[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:35 dist-test-slave-sb2z krb5kdc[31039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:35 dist-test-slave-sb2z krb5kdc[31039](info): set up 2 sockets
Aug 08 15:51:35 dist-test-slave-sb2z krb5kdc[31039](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 15:51:36.243206 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:36 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:36.364193 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39451
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:36.476075 31059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:36.476336 31059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:36.476375 31059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:36.480440 31059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:36.480496 31059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:36.480515 31059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:36.480566 31059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:36.480602 31059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:36.485797 31059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:36.486825 31059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:36.487866 31059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:36.496603 31068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:36.496590 31069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:36.496594 31071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:36.496757 31059 server_base.cc:1034] running on GCE node
I20260808 15:51:36.497532 31059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:36.498608 31059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:36.499891 31059 hybrid_clock.cc:648] HybridClock initialized: now 1786204296499876 us; error 53 us; skew 500 ppm
Aug 08 15:51:36 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:36.502686 31059 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:36.503639 31059 webserver.cc:460] Webserver started at http://127.30.76.126:36013/ using document root <none> and password file <none>
I20260808 15:51:36.504206 31059 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:36.504311 31059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:36.504545 31059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:36.506600 31059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "a165677319e54eb49cdcaadf239f3b4c"
format_stamp: "Formatted at 2026-08-08 15:51:36 on dist-test-slave-sb2z"
server_key: "63e08ce452f631ef495a838301165a4f"
server_key_iv: "b4b0b28018b66feff8167d4ea76a2f23"
server_key_version: "encryptionkey@0"
I20260808 15:51:36.507169 31059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "a165677319e54eb49cdcaadf239f3b4c"
format_stamp: "Formatted at 2026-08-08 15:51:36 on dist-test-slave-sb2z"
server_key: "63e08ce452f631ef495a838301165a4f"
server_key_iv: "b4b0b28018b66feff8167d4ea76a2f23"
server_key_version: "encryptionkey@0"
I20260808 15:51:36.510846 31059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:51:36.513173 31081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:36.514319 31059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:36.514503 31059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "a165677319e54eb49cdcaadf239f3b4c"
format_stamp: "Formatted at 2026-08-08 15:51:36 on dist-test-slave-sb2z"
server_key: "63e08ce452f631ef495a838301165a4f"
server_key_iv: "b4b0b28018b66feff8167d4ea76a2f23"
server_key_version: "encryptionkey@0"
I20260808 15:51:36.514613 31059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:36.526973 31059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:36.529619 31059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:36.529819 31059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:36.540084 31059 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39451
I20260808 15:51:36.540126 31159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39451 every 8 connection(s)
I20260808 15:51:36.541178 31059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:36.543939 31163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:36.549324 31163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c: Bootstrap starting.
I20260808 15:51:36.550769 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31059
I20260808 15:51:36.550932 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:36.552405 31163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:36.552829 31025 external_mini_cluster.cc:1428] Setting key 49caa6ce78dc1bc56370a9a92b3c7065
I20260808 15:51:36.553318 31163 log.cc:824] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:36.555671 31163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c: No bootstrap required, opened a new log
I20260808 15:51:36.561710 31163 raft_consensus.cc:348] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165677319e54eb49cdcaadf239f3b4c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39451 } }
I20260808 15:51:36.562044 31163 raft_consensus.cc:374] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:36.562126 31163 raft_consensus.cc:729] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a165677319e54eb49cdcaadf239f3b4c, State: Initialized, Role: FOLLOWER
I20260808 15:51:36.562680 31163 consensus_queue.cc:260] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165677319e54eb49cdcaadf239f3b4c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39451 } }
I20260808 15:51:36.562867 31163 raft_consensus.cc:388] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:36.562950 31163 raft_consensus.cc:482] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:36.563052 31163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:36.564435 31163 raft_consensus.cc:504] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165677319e54eb49cdcaadf239f3b4c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39451 } }
I20260808 15:51:36.564796 31163 leader_election.cc:302] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a165677319e54eb49cdcaadf239f3b4c; no voters:
I20260808 15:51:36.565146 31163 leader_election.cc:288] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:36.565289 31168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:36.565619 31168 raft_consensus.cc:686] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [term 1 LEADER]: Becoming Leader. State: Replica: a165677319e54eb49cdcaadf239f3b4c, State: Running, Role: LEADER
I20260808 15:51:36.565927 31168 consensus_queue.cc:237] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165677319e54eb49cdcaadf239f3b4c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39451 } }
I20260808 15:51:36.566499 31163 sys_catalog.cc:564] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:36.567754 31173 sys_catalog.cc:455] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a165677319e54eb49cdcaadf239f3b4c. Latest consensus state: current_term: 1 leader_uuid: "a165677319e54eb49cdcaadf239f3b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165677319e54eb49cdcaadf239f3b4c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39451 } } }
I20260808 15:51:36.567725 31172 sys_catalog.cc:455] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a165677319e54eb49cdcaadf239f3b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165677319e54eb49cdcaadf239f3b4c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39451 } } }
I20260808 15:51:36.567934 31172 sys_catalog.cc:458] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:36.567934 31173 sys_catalog.cc:458] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:36.568625 31177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:36.572378 31177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 15:51:36 dist-test-slave-sb2z krb5kdc[31039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:36.578761 31177 catalog_manager.cc:1347] Generated new cluster ID: eeb182a15e28483db58881bf3aad9bfa
I20260808 15:51:36.578820 31177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:36.584986 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:36.556231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.30.76.126:39451)
0808 15:51:36.556820 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:36.556869 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:36.557893 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 15:51:36.563223 (+ 5330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:36.563236 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:36.563507 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:36.563816 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:36.567287 (+ 3471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:36.569056 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:36.570780 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:36.571598 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:36.571755 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:36.577033 (+ 5278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:36.577041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:36.577058 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:36.577115 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:36.580785 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:36.581395 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:36.581402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:36.581407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:36.581585 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:36.581965 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:36.581968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:36.581970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:36.582698 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:36.583020 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:36.583500 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 15:51:36.583820 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":178,"threads_started":1}
I20260808 15:51:36.615933 31177 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:36.617017 31177 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:36.624243 31177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a165677319e54eb49cdcaadf239f3b4c: Generated new TSK 0
I20260808 15:51:36.625141 31177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:36.666373 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39451
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:36.782964 31203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:36.783207 31203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:36.783282 31203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:36.787233 31203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:36.787287 31203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:36.787405 31203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:36.792527 31203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:36.793506 31203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:36.794510 31203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:36.804358 31210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:36.804457 31212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:36.804353 31209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:36.804638 31203 server_base.cc:1034] running on GCE node
I20260808 15:51:36.805080 31203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:36.805708 31203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:36.806903 31203 hybrid_clock.cc:648] HybridClock initialized: now 1786204296806871 us; error 49 us; skew 500 ppm
Aug 08 15:51:36 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:36.809722 31203 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:36.810557 31203 webserver.cc:460] Webserver started at http://127.30.76.65:41275/ using document root <none> and password file <none>
I20260808 15:51:36.811117 31203 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:36.811218 31203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:36.811450 31203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:36.813434 31203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "b02ebdd6c8b549f995bf216008873283"
format_stamp: "Formatted at 2026-08-08 15:51:36 on dist-test-slave-sb2z"
server_key: "95667378ead53af8b018f429c8e18343"
server_key_iv: "452aa3004be8c99952dd63f8a54d0c0d"
server_key_version: "encryptionkey@0"
I20260808 15:51:36.813993 31203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "b02ebdd6c8b549f995bf216008873283"
format_stamp: "Formatted at 2026-08-08 15:51:36 on dist-test-slave-sb2z"
server_key: "95667378ead53af8b018f429c8e18343"
server_key_iv: "452aa3004be8c99952dd63f8a54d0c0d"
server_key_version: "encryptionkey@0"
I20260808 15:51:36.817798 31203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:51:36.820331 31221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:36.821537 31203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:51:36.821674 31203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "b02ebdd6c8b549f995bf216008873283"
format_stamp: "Formatted at 2026-08-08 15:51:36 on dist-test-slave-sb2z"
server_key: "95667378ead53af8b018f429c8e18343"
server_key_iv: "452aa3004be8c99952dd63f8a54d0c0d"
server_key_version: "encryptionkey@0"
I20260808 15:51:36.821785 31203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:36.850579 31203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:36.854473 31203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:36.854750 31203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:36.855597 31203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:36.856555 31203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:36.856630 31203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:36.856706 31203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:36.856750 31203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:36.866477 31203 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:33661
I20260808 15:51:36.866533 31385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:33661 every 8 connection(s)
I20260808 15:51:36.867444 31203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:36.873435 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31203
I20260808 15:51:36.873567 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:36.873888 31025 external_mini_cluster.cc:1428] Setting key bf4c5952c0ff10d29a32de03e2cba969
Aug 08 15:51:36 dist-test-slave-sb2z krb5kdc[31039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:36.882707 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:36.869479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:34603 (local address 127.30.76.126:39451)
0808 15:51:36.869616 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:36.869620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:36.870704 (+ 1084us) server_negotiation.cc:407] Connection header received
0808 15:51:36.871660 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:36.871663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:36.871730 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:36.871833 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:36.872996 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:36.873539 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:36.875091 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:36.875427 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:36.875559 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:36.878476 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:36.878479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:36.878481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:36.878504 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:36.880359 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:36.880898 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:36.880904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:36.880909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:36.881027 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:36.881432 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:36.881436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:36.881437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:36.881687 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:36.881803 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:36.882300 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 15:51:36.882451 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:51:36.883386 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:36.869822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39451 (local address 127.30.76.65:34603)
0808 15:51:36.870554 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:36.870594 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:36.871387 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:36.872017 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:36.872024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:36.872375 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:36.872843 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:36.872857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:36.873749 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:36.873754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:36.874962 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:36.874972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:36.875618 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:36.875623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:36.875838 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:36.876425 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:36.876451 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:36.878288 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:36.880497 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:36.880503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:36.880514 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:36.880783 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:36.881164 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:36.881168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:36.881171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:36.881327 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:36.881809 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:36.881815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:36.882048 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 15:51:36.882209 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 15:51:36.882471 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":114,"threads_started":1}
I20260808 15:51:36.884783 31386 heartbeater.cc:347] Connected to a master server at 127.30.76.126:39451
I20260808 15:51:36.885051 31386 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:36.885540 31386 heartbeater.cc:511] Master 127.30.76.126:39451 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:51:36.887207 31107 ts_manager.cc:194] Registered new tserver with Master: b02ebdd6c8b549f995bf216008873283 (127.30.76.65:33661)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:51:36.888707 31107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:34603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:36.903993 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39451
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:37.018913 31394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:37.019239 31394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:37.019320 31394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:37.023427 31394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:37.023514 31394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:37.023630 31394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:37.029131 31394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:37.030243 31394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:37.031265 31394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:37.040879 31402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:37.040901 31401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:37.040930 31404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:37.041209 31394 server_base.cc:1034] running on GCE node
I20260808 15:51:37.041751 31394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:37.042388 31394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:37.043609 31394 hybrid_clock.cc:648] HybridClock initialized: now 1786204297043599 us; error 30 us; skew 500 ppm
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:37.046355 31394 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:37.047173 31394 webserver.cc:460] Webserver started at http://127.30.76.66:42123/ using document root <none> and password file <none>
I20260808 15:51:37.047781 31394 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:37.047881 31394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:37.048116 31394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:37.050041 31394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "7aad2bc810bd4ff097b372eacddb4d33"
format_stamp: "Formatted at 2026-08-08 15:51:37 on dist-test-slave-sb2z"
server_key: "a4d1240d314f3fc718a84311190c91de"
server_key_iv: "c3d2f0838651786ba9832b954019037a"
server_key_version: "encryptionkey@0"
I20260808 15:51:37.050596 31394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "7aad2bc810bd4ff097b372eacddb4d33"
format_stamp: "Formatted at 2026-08-08 15:51:37 on dist-test-slave-sb2z"
server_key: "a4d1240d314f3fc718a84311190c91de"
server_key_iv: "c3d2f0838651786ba9832b954019037a"
server_key_version: "encryptionkey@0"
I20260808 15:51:37.054214 31394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 15:51:37.056530 31414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:37.057698 31394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:37.057847 31394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "7aad2bc810bd4ff097b372eacddb4d33"
format_stamp: "Formatted at 2026-08-08 15:51:37 on dist-test-slave-sb2z"
server_key: "a4d1240d314f3fc718a84311190c91de"
server_key_iv: "c3d2f0838651786ba9832b954019037a"
server_key_version: "encryptionkey@0"
I20260808 15:51:37.057957 31394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:37.069054 31394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:37.072414 31394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:37.072619 31394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:37.073196 31394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:37.074096 31394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:37.074168 31394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:37.074240 31394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:37.074291 31394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:37.083158 31394 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:40333
I20260808 15:51:37.083499 31584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:40333 every 8 connection(s)
I20260808 15:51:37.084164 31394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:37.091054 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31394
I20260808 15:51:37.091144 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:37.091377 31025 external_mini_cluster.cc:1428] Setting key 8efb0e271b6515ed3282693b3326bbf4
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:37.098212 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:37.086147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:37227 (local address 127.30.76.126:39451)
0808 15:51:37.086283 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:37.086286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:37.087118 (+ 832us) server_negotiation.cc:407] Connection header received
0808 15:51:37.088078 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:37.088081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:37.088140 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:37.088223 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:37.089343 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.089890 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.091037 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.091338 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.091452 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:37.094015 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:37.094019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:37.094021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:37.094043 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:37.095907 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.096550 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.096552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.096554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.096635 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.097077 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.097080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.097081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.097263 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:37.097336 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:37.097981 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 15:51:37.098079 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:51:37.099401 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:37.086533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39451 (local address 127.30.76.66:37227)
0808 15:51:37.086975 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:37.087017 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:37.087871 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:37.088386 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:37.088393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:37.088738 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:37.089197 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:37.089211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.090021 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.090026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:37.090908 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:37.090918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.091500 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.091504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:37.091698 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:37.092286 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:37.092302 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:37.093848 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:37.096027 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:37.096035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:37.096050 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:37.096445 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:37.096762 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:37.096769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:37.096773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:37.096969 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:37.097371 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:37.097380 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:37.097721 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 15:51:37.097953 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 15:51:37.098348 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":112,"threads_started":1}
I20260808 15:51:37.100610 31585 heartbeater.cc:347] Connected to a master server at 127.30.76.126:39451
I20260808 15:51:37.100858 31585 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:37.101354 31585 heartbeater.cc:511] Master 127.30.76.126:39451 requested a full tablet report, sending...
I20260808 15:51:37.102473 31107 ts_manager.cc:194] Registered new tserver with Master: 7aad2bc810bd4ff097b372eacddb4d33 (127.30.76.66:40333)
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:51:37.103258 31107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:37227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:37.120167 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39451
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:37.231245 31591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:37.231551 31591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:37.231630 31591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:37.235555 31591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:37.235639 31591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:37.235728 31591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:37.240847 31591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:37.241870 31591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:37.242861 31591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:37.252278 31605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:37.252467 31599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:37.252493 31602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:37.252633 31591 server_base.cc:1034] running on GCE node
I20260808 15:51:37.253067 31591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:37.253662 31591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:37.254853 31591 hybrid_clock.cc:648] HybridClock initialized: now 1786204297254842 us; error 31 us; skew 500 ppm
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:37.258337 31591 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:37.259478 31591 webserver.cc:460] Webserver started at http://127.30.76.67:38029/ using document root <none> and password file <none>
I20260808 15:51:37.260325 31591 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:37.260457 31591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:37.260746 31591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:37.263764 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "f2b663e4f9e14782860620e4666b25c0"
format_stamp: "Formatted at 2026-08-08 15:51:37 on dist-test-slave-sb2z"
server_key: "565fd5b89560ef4a77519f94ce54fab7"
server_key_iv: "996eaede7e62b7b44f9cc703aa6abe2d"
server_key_version: "encryptionkey@0"
I20260808 15:51:37.264475 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "f2b663e4f9e14782860620e4666b25c0"
format_stamp: "Formatted at 2026-08-08 15:51:37 on dist-test-slave-sb2z"
server_key: "565fd5b89560ef4a77519f94ce54fab7"
server_key_iv: "996eaede7e62b7b44f9cc703aa6abe2d"
server_key_version: "encryptionkey@0"
I20260808 15:51:37.269527 31591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 15:51:37.272794 31623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:37.274341 31591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 15:51:37.274484 31591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "f2b663e4f9e14782860620e4666b25c0"
format_stamp: "Formatted at 2026-08-08 15:51:37 on dist-test-slave-sb2z"
server_key: "565fd5b89560ef4a77519f94ce54fab7"
server_key_iv: "996eaede7e62b7b44f9cc703aa6abe2d"
server_key_version: "encryptionkey@0"
I20260808 15:51:37.274610 31591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:37.298272 31591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:37.302186 31591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:37.302397 31591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:37.303188 31591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:37.304404 31591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:37.304474 31591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:37.304541 31591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:37.304582 31591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:37.317705 31591 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:33515
I20260808 15:51:37.317876 31784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:33515 every 8 connection(s)
I20260808 15:51:37.319150 31591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:37.326952 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31591
I20260808 15:51:37.327062 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:37.327319 31025 external_mini_cluster.cc:1428] Setting key 7c75ff92bf4ac5605d7bb5bee47ed09d
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:37.344313 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:37.329833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:34079 (local address 127.30.76.126:39451)
0808 15:51:37.329955 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:37.329960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:37.331296 (+ 1336us) server_negotiation.cc:407] Connection header received
0808 15:51:37.332265 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:37.332268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:37.332322 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:37.332399 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:37.333763 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.334773 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.336573 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.337021 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.337161 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:37.340156 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:37.340159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:37.340161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:37.340185 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:37.341953 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.342748 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.342751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.342753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.342847 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.343200 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.343203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.343204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.343392 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:37.343500 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:37.344010 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 15:51:37.344145 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:51:37.345343 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:37.330387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39451 (local address 127.30.76.67:34079)
0808 15:51:37.331152 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:37.331204 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:37.332070 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:37.332566 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:37.332572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:37.332922 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:37.333585 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:37.333606 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.334927 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.334931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:37.336438 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:37.336450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.337193 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.337197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:37.337375 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:37.337995 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:37.338011 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:37.339969 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:37.342275 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:37.342279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:37.342291 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:37.342640 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:37.342939 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:37.342943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:37.342945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:37.343092 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:37.343498 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:37.343505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:37.343769 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 15:51:37.343949 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 15:51:37.344265 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":160,"threads_started":1}
I20260808 15:51:37.346498 31786 heartbeater.cc:347] Connected to a master server at 127.30.76.126:39451
I20260808 15:51:37.346809 31786 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:37.347465 31786 heartbeater.cc:511] Master 127.30.76.126:39451 requested a full tablet report, sending...
I20260808 15:51:37.348670 31107 ts_manager.cc:194] Registered new tserver with Master: f2b663e4f9e14782860620e4666b25c0 (127.30.76.67:33515)
I20260808 15:51:37.349232 31107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:34079
I20260808 15:51:37.354329 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:37.386070 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:37.374176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.30.76.65:33661)
0808 15:51:37.374472 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:37.374491 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:37.374537 (+ 46us) server_negotiation.cc:407] Connection header received
0808 15:51:37.374595 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:37.374600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:37.374784 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:37.374898 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:37.375899 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.376672 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.378116 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.378468 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.378548 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:37.381655 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:37.381670 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:37.381689 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:37.381728 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:37.383905 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.384344 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.384354 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.384357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.384471 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.384784 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.384789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.384792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.385196 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:37.385440 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:37.385588 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 15:51:37.385775 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":106,"threads_started":1}
W20260808 15:51:37.387068 31262 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54426
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:37.414669 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:37.405070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.30.76.65:33661)
0808 15:51:37.405240 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:37.405247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:37.405346 (+ 99us) server_negotiation.cc:407] Connection header received
0808 15:51:37.405459 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:37.405464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:37.405519 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:37.405618 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:37.406285 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.407210 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.408090 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:37.408543 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:37.408679 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:37.411015 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:37.411022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:37.411027 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:37.411064 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:37.412913 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.413317 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.413323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.413327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.413429 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:37.413687 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:37.413693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:37.413705 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:37.413940 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:37.414109 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:37.414267 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 15:51:37.414434 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:51:37.415859 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31203
I20260808 15:51:37.430004 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31394
I20260808 15:51:37.443307 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31591
I20260808 15:51:37.452940 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31059
2026-08-08T15:51:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2467 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31809](info): set up 2 sockets
Aug 08 15:51:37 dist-test-slave-sb2z krb5kdc[31809](info): commencing operation
krb5kdc: starting...
W20260808 15:51:38.659036 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:38 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:38.801726 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:34789
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:34789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:38.912186 31827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:38.912442 31827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:38.912482 31827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:38.917687 31827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:38.917750 31827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:38.917769 31827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:38.917786 31827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:38.917801 31827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:38.922950 31827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:34789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:34789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:38.924039 31827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:38.924969 31827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:38.933262 31840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:38.933285 31837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:38.933283 31843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:38.933606 31827 server_base.cc:1034] running on GCE node
I20260808 15:51:38.934221 31827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:38.935276 31827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:38.936466 31827 hybrid_clock.cc:648] HybridClock initialized: now 1786204298936443 us; error 37 us; skew 500 ppm
Aug 08 15:51:38 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204298, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:38.939121 31827 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:38.940061 31827 webserver.cc:460] Webserver started at http://127.30.76.126:36145/ using document root <none> and password file <none>
I20260808 15:51:38.940608 31827 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:38.940717 31827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:38.940961 31827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:38.943032 31827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "904298beeb3f4044a70c0ba5e8f669ba"
format_stamp: "Formatted at 2026-08-08 15:51:38 on dist-test-slave-sb2z"
server_key: "319dc35f07a3efb1e22676da3d054bb2"
server_key_iv: "97b86da9f5fdd3d6ed7bf991269702e7"
server_key_version: "encryptionkey@0"
I20260808 15:51:38.943692 31827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "904298beeb3f4044a70c0ba5e8f669ba"
format_stamp: "Formatted at 2026-08-08 15:51:38 on dist-test-slave-sb2z"
server_key: "319dc35f07a3efb1e22676da3d054bb2"
server_key_iv: "97b86da9f5fdd3d6ed7bf991269702e7"
server_key_version: "encryptionkey@0"
I20260808 15:51:38.947190 31827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:51:38.949491 31854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:38.950637 31827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:38.950793 31827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "904298beeb3f4044a70c0ba5e8f669ba"
format_stamp: "Formatted at 2026-08-08 15:51:38 on dist-test-slave-sb2z"
server_key: "319dc35f07a3efb1e22676da3d054bb2"
server_key_iv: "97b86da9f5fdd3d6ed7bf991269702e7"
server_key_version: "encryptionkey@0"
I20260808 15:51:38.950903 31827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:38.990101 31827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:38.993001 31827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:38.993217 31827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:39.000836 31827 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:34789
I20260808 15:51:39.000864 31924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:34789 every 8 connection(s)
I20260808 15:51:39.001844 31827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:39.004678 31927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:39.007887 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31827
I20260808 15:51:39.007978 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:39.008239 31025 external_mini_cluster.cc:1428] Setting key 1bb7e9752d89c59bc80c5cf0172f6198
I20260808 15:51:39.010975 31927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba: Bootstrap starting.
I20260808 15:51:39.013273 31927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:39.014039 31927 log.cc:824] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:39.016109 31927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba: No bootstrap required, opened a new log
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:39.019567 31927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904298beeb3f4044a70c0ba5e8f669ba" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34789 } }
I20260808 15:51:39.019799 31927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:39.019872 31927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 904298beeb3f4044a70c0ba5e8f669ba, State: Initialized, Role: FOLLOWER
I20260808 15:51:39.020361 31927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904298beeb3f4044a70c0ba5e8f669ba" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34789 } }
I20260808 15:51:39.020533 31927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:39.020604 31927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:39.020709 31927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:39.021586 31927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904298beeb3f4044a70c0ba5e8f669ba" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34789 } }
I20260808 15:51:39.021973 31927 leader_election.cc:302] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 904298beeb3f4044a70c0ba5e8f669ba; no voters:
I20260808 15:51:39.022351 31927 leader_election.cc:288] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:39.022526 31936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:39.022756 31936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [term 1 LEADER]: Becoming Leader. State: Replica: 904298beeb3f4044a70c0ba5e8f669ba, State: Running, Role: LEADER
I20260808 15:51:39.023154 31936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904298beeb3f4044a70c0ba5e8f669ba" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34789 } }
I20260808 15:51:39.023559 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.009735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.30.76.126:34789)
0808 15:51:39.010196 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.010212 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.010250 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:51:39.010846 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.010862 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.011244 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.011606 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.012419 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.013426 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.014304 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.014770 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.014943 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.017750 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.017755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.017768 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.017806 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.020213 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.020778 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.020786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.020794 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.020935 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.021227 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.021231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.021233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.021619 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.021847 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.022181 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.022521 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":130,"threads_started":1}
I20260808 15:51:39.023741 31927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:39.024431 31938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 904298beeb3f4044a70c0ba5e8f669ba. Latest consensus state: current_term: 1 leader_uuid: "904298beeb3f4044a70c0ba5e8f669ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904298beeb3f4044a70c0ba5e8f669ba" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34789 } } }
I20260808 15:51:39.024550 31938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:39.025218 31939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "904298beeb3f4044a70c0ba5e8f669ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904298beeb3f4044a70c0ba5e8f669ba" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34789 } } }
I20260808 15:51:39.025314 31939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba [sys.catalog]: This master's current role is: LEADER
W20260808 15:51:39.028687 31957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:51:39.028780 31957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:51:39.028900 31944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:39.029834 31944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:39.035247 31944 catalog_manager.cc:1347] Generated new cluster ID: 50813ba7bc2e426888c0144fd9988d26
I20260808 15:51:39.035316 31944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:39.056387 31944 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:39.057647 31944 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:39.066959 31944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 904298beeb3f4044a70c0ba5e8f669ba: Generated new TSK 0
I20260808 15:51:39.067684 31944 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:39.104305 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34789
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:39.215499 31963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:39.215773 31963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:39.215850 31963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:39.220048 31963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:39.220104 31963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:39.220163 31963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:39.225854 31963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:39.226945 31963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:39.228058 31963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:39.237963 31977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:39.238029 31974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:39.237996 31981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:39.238082 31963 server_base.cc:1034] running on GCE node
I20260808 15:51:39.238742 31963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:39.239377 31963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:39.240566 31963 hybrid_clock.cc:648] HybridClock initialized: now 1786204299240537 us; error 50 us; skew 500 ppm
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:39.243469 31963 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:39.244421 31963 webserver.cc:460] Webserver started at http://127.30.76.65:33601/ using document root <none> and password file <none>
I20260808 15:51:39.245102 31963 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:39.245193 31963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:39.245415 31963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:39.247359 31963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "6df2a18d75144994958759b04141cc12"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "b4eeef31a2d85d83b7cd06b3deda170d"
server_key_iv: "8bfb2cdb484c2d6ec92fc43fa4f87af9"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.248019 31963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "6df2a18d75144994958759b04141cc12"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "b4eeef31a2d85d83b7cd06b3deda170d"
server_key_iv: "8bfb2cdb484c2d6ec92fc43fa4f87af9"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.251576 31963 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:51:39.253824 31996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.255028 31963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:51:39.255158 31963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "6df2a18d75144994958759b04141cc12"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "b4eeef31a2d85d83b7cd06b3deda170d"
server_key_iv: "8bfb2cdb484c2d6ec92fc43fa4f87af9"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.255250 31963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:39.272771 31963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:39.275482 31963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:39.275691 31963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:39.276271 31963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:39.277067 31963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:39.277114 31963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.277177 31963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:39.277212 31963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.286118 31963 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:34883
I20260808 15:51:39.286150 32154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:34883 every 8 connection(s)
I20260808 15:51:39.286960 31963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:39.290975 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31963
I20260808 15:51:39.291101 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:39.291366 31025 external_mini_cluster.cc:1428] Setting key 9ec4c51b88f277a99de72c99f4f03d27
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:39.300753 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.288719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:42333 (local address 127.30.76.126:34789)
0808 15:51:39.288882 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.288887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.289693 (+ 806us) server_negotiation.cc:407] Connection header received
0808 15:51:39.290480 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.290482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.290532 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.290607 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.291907 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.292547 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.293747 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.294153 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.294279 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.297081 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.297084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.297086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.297109 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.298507 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.299157 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.299159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.299160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.299230 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.299675 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.299678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.299679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.299845 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.299922 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.300530 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.300634 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:51:39.301955 32163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.289107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:34789 (local address 127.30.76.65:42333)
0808 15:51:39.289534 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.289575 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.290293 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.290774 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.290780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.291150 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:39.291733 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.291749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.292708 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.292712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.293612 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.293622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.294300 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.294303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.294431 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.295044 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:39.295060 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:39.296915 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:39.298634 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:39.298644 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:39.298662 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:39.299051 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:39.299327 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:39.299332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:39.299336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:39.299576 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:39.299955 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:39.299963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:39.300278 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.300484 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.300851 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:51:39.303202 32156 heartbeater.cc:347] Connected to a master server at 127.30.76.126:34789
I20260808 15:51:39.303507 32156 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:39.304028 32156 heartbeater.cc:511] Master 127.30.76.126:34789 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:51:39.305630 31875 ts_manager.cc:194] Registered new tserver with Master: 6df2a18d75144994958759b04141cc12 (127.30.76.65:34883)
I20260808 15:51:39.306749 31875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:42333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:39.320523 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34789
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:39.433274 32166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:39.433554 32166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:39.433637 32166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:39.437937 32166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:39.438030 32166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:39.438117 32166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:39.443403 32166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:39.444526 32166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:39.445540 32166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:39.455173 32173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:39.455225 32176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:39.455188 32172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:39.455585 32166 server_base.cc:1034] running on GCE node
I20260808 15:51:39.455935 32166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:39.456507 32166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:39.457676 32166 hybrid_clock.cc:648] HybridClock initialized: now 1786204299457658 us; error 26 us; skew 500 ppm
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:39.460556 32166 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:39.461331 32166 webserver.cc:460] Webserver started at http://127.30.76.66:42113/ using document root <none> and password file <none>
I20260808 15:51:39.461844 32166 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:39.461903 32166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:39.462059 32166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:39.464016 32166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "3b78401091514575b522a4340f18103f"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "e43afd0ee1f51e319b1600f70a8f9b07"
server_key_iv: "02898b611dc3a8464a4b01c1e8a392fc"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.464515 32166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "3b78401091514575b522a4340f18103f"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "e43afd0ee1f51e319b1600f70a8f9b07"
server_key_iv: "02898b611dc3a8464a4b01c1e8a392fc"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.468050 32166 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:39.470259 32184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.471410 32166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:39.471532 32166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3b78401091514575b522a4340f18103f"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "e43afd0ee1f51e319b1600f70a8f9b07"
server_key_iv: "02898b611dc3a8464a4b01c1e8a392fc"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.471678 32166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:39.489656 32166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:39.492537 32166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:39.492740 32166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:39.493335 32166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:39.494160 32166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:39.494232 32166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.494305 32166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:39.494357 32166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.503270 32166 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:45527
I20260808 15:51:39.503366 32347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:45527 every 8 connection(s)
I20260808 15:51:39.504205 32166 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:39.507853 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 32166
I20260808 15:51:39.507983 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:39.508244 31025 external_mini_cluster.cc:1428] Setting key ce10d724cbdf341bb13c2add20a5b12d
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:39.518527 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.506001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56815 (local address 127.30.76.126:34789)
0808 15:51:39.506148 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.506153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.506950 (+ 797us) server_negotiation.cc:407] Connection header received
0808 15:51:39.507855 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.507859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.507918 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.508021 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.509434 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.510181 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.511285 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.511704 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.511813 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.514423 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.514430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.514437 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.514470 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.516393 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.516903 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.516910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.516913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.517011 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.517361 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.517365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.517367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.517624 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.517736 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.518197 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.518330 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:51:39.519218 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.506350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:34789 (local address 127.30.76.66:56815)
0808 15:51:39.506800 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.506840 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.507648 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.508196 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.508202 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.508641 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:39.509269 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.509288 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.510313 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.510316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.511159 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.511168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.511867 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.511873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.512086 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.512691 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:39.512714 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:39.514262 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:39.516513 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:39.516518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:39.516529 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:39.516798 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:39.517116 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:39.517118 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:39.517120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:39.517261 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:39.517731 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:39.517735 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:39.517948 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.518095 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.518332 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:51:39.520409 32348 heartbeater.cc:347] Connected to a master server at 127.30.76.126:34789
I20260808 15:51:39.520612 32348 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:39.521114 32348 heartbeater.cc:511] Master 127.30.76.126:34789 requested a full tablet report, sending...
I20260808 15:51:39.522087 31875 ts_manager.cc:194] Registered new tserver with Master: 3b78401091514575b522a4340f18103f (127.30.76.66:45527)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:51:39.522891 31875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:56815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:39.536975 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34789
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:39.649796 32354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:39.650032 32354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:39.650074 32354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:39.654083 32354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:39.654150 32354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:39.654206 32354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:39.659482 32354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:39.660379 32354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:39.661316 32354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:39.670855 32367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:39.670859 32361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:39.670859 32362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:39.671191 32354 server_base.cc:1034] running on GCE node
I20260808 15:51:39.671630 32354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:39.672199 32354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:39.673501 32354 hybrid_clock.cc:648] HybridClock initialized: now 1786204299673472 us; error 39 us; skew 500 ppm
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:39.676218 32354 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:39.677076 32354 webserver.cc:460] Webserver started at http://127.30.76.67:36743/ using document root <none> and password file <none>
I20260808 15:51:39.677625 32354 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:39.677721 32354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:39.677945 32354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:39.680059 32354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "8df1842515374833b6b9d885bd0af73a"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "7a18f97bb0f02f9e52896ebcd1bb3dad"
server_key_iv: "f778ee17cc3b741929fe9a32985b737c"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.680620 32354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "8df1842515374833b6b9d885bd0af73a"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "7a18f97bb0f02f9e52896ebcd1bb3dad"
server_key_iv: "f778ee17cc3b741929fe9a32985b737c"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.684149 32354 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:51:39.686518 32379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.687697 32354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 15:51:39.687824 32354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "8df1842515374833b6b9d885bd0af73a"
format_stamp: "Formatted at 2026-08-08 15:51:39 on dist-test-slave-sb2z"
server_key: "7a18f97bb0f02f9e52896ebcd1bb3dad"
server_key_iv: "f778ee17cc3b741929fe9a32985b737c"
server_key_version: "encryptionkey@0"
I20260808 15:51:39.687928 32354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:39.705897 32354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:39.708724 32354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:39.708933 32354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:39.709534 32354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:39.710364 32354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:39.710438 32354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.710517 32354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:39.710563 32354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:39.720193 32354 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:34535
I20260808 15:51:39.720229 32524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:34535 every 8 connection(s)
I20260808 15:51:39.721097 32354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:39.723474 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 32354
I20260808 15:51:39.723641 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:39.723985 31025 external_mini_cluster.cc:1428] Setting key 5032d3519ada05b478a34496fb911787
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:39.734138 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.722939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:43375 (local address 127.30.76.126:34789)
0808 15:51:39.723097 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.723103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.724207 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 15:51:39.725460 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.725463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.725520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.725598 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.726803 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.727326 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.728491 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.728723 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.728804 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.731065 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.731068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.731070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.731099 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.732393 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.732854 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.732856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.732857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.732933 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.733232 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.733234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.733235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.733404 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.733479 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.733919 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.734014 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:51:39.734882 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.723384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:34789 (local address 127.30.76.67:43375)
0808 15:51:39.724026 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.724090 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.725184 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.725733 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.725739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.726193 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:39.726677 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.726690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.727432 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.727435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.728386 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.728396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.728851 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.728854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.728988 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.729496 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:39.729512 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:39.730922 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:39.732496 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:39.732500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:39.732511 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:39.732763 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:39.733016 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:39.733018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:39.733020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:39.733148 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:39.733491 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:39.733496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:39.733713 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.733862 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.734088 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":197,"threads_started":1}
I20260808 15:51:39.736045 32525 heartbeater.cc:347] Connected to a master server at 127.30.76.126:34789
I20260808 15:51:39.736289 32525 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:39.736842 32525 heartbeater.cc:511] Master 127.30.76.126:34789 requested a full tablet report, sending...
I20260808 15:51:39.738042 31874 ts_manager.cc:194] Registered new tserver with Master: 8df1842515374833b6b9d885bd0af73a (127.30.76.67:34535)
I20260808 15:51:39.738662 31874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:43375
I20260808 15:51:39.749900 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:51:39.761288 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.753841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.30.76.126:34789)
0808 15:51:39.754021 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.754025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.754137 (+ 112us) server_negotiation.cc:407] Connection header received
0808 15:51:39.754274 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.754276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.754320 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.754388 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.755163 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.755709 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.756482 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.756697 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.756771 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.758077 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.758080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.758082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.758106 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.759449 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.759948 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.759952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.759954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.760042 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.760350 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.760354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.760355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.760588 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.760743 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.760999 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.761117 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 15:51:39.766326 31874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:51:39.769593 31874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:39.783960 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.778481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.30.76.67:34535)
0808 15:51:39.778835 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.778850 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.778911 (+ 61us) server_negotiation.cc:407] Connection header received
0808 15:51:39.779208 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.779213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.779422 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.779615 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:39.779936 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.781059 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.782462 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.783232 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.783353 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.783495 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.783707 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.783797 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":134,"threads_started":1}
I20260808 15:51:39.783953 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.778291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:34535 (local address 127.0.0.1:39074)
0808 15:51:39.778776 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.778830 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.778980 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.779601 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.779607 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.779632 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:39.779789 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.779797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.781205 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.781210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.782322 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.782333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.783366 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.783369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.783477 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.783515 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.783674 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.783797 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":99,"threads_started":1}
I20260808 15:51:39.784475 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.779047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:34883 (local address 127.0.0.1:59382)
0808 15:51:39.779323 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.779340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.779424 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.780376 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.780379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.780398 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:39.780502 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.780509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.781795 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.781798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.782893 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.782903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.784094 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.784098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.784184 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.784206 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.784282 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.784345 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":119,"threads_started":1}
I20260808 15:51:39.784567 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.779190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.30.76.65:34883)
0808 15:51:39.779934 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.779953 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.779977 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:51:39.780058 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.780063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.780238 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.780397 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:39.780624 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.781652 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.783002 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.783802 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.783900 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.784032 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.784342 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.784434 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":90,"threads_started":1}
I20260808 15:51:39.785233 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.779405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:45527 (local address 127.0.0.1:59350)
0808 15:51:39.780413 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.780425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.780512 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.781008 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.781011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.781031 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:39.781115 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.781121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.782345 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.782348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.783752 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.783765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.784858 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.784862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.784946 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.784968 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.785047 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.785108 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":103,"threads_started":1}
I20260808 15:51:39.785269 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.779780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.30.76.66:45527)
0808 15:51:39.780274 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.780284 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.780597 (+ 313us) server_negotiation.cc:407] Connection header received
0808 15:51:39.780684 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.780689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.780845 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.780986 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:39.781258 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.782154 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.783903 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.784728 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.784832 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.784925 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.785082 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.785158 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":105,"threads_started":1}
I20260808 15:51:39.787669 32439 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6ca24a65c14df5af97ad6bde917977 (DEFAULT_TABLE table=test-table [id=debec0b710d5472ca1b20b6eab40758a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:39.787978 32060 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6ca24a65c14df5af97ad6bde917977 (DEFAULT_TABLE table=test-table [id=debec0b710d5472ca1b20b6eab40758a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:39.788833 32439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d6ca24a65c14df5af97ad6bde917977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:39.788875 32060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d6ca24a65c14df5af97ad6bde917977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:39.789885 32256 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6ca24a65c14df5af97ad6bde917977 (DEFAULT_TABLE table=test-table [id=debec0b710d5472ca1b20b6eab40758a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:39.790788 32256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d6ca24a65c14df5af97ad6bde917977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:39.796147 32555 tablet_bootstrap.cc:492] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12: Bootstrap starting.
I20260808 15:51:39.796893 32556 tablet_bootstrap.cc:492] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a: Bootstrap starting.
I20260808 15:51:39.798401 32555 tablet_bootstrap.cc:654] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:39.798573 32556 tablet_bootstrap.cc:654] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:39.798936 32557 tablet_bootstrap.cc:492] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f: Bootstrap starting.
I20260808 15:51:39.799695 32556 log.cc:824] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:39.800163 32555 log.cc:824] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:39.801332 32557 tablet_bootstrap.cc:654] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:39.801983 32557 log.cc:824] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:39.802201 32556 tablet_bootstrap.cc:492] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a: No bootstrap required, opened a new log
I20260808 15:51:39.802201 32555 tablet_bootstrap.cc:492] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12: No bootstrap required, opened a new log
I20260808 15:51:39.802397 32556 ts_tablet_manager.cc:1397] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 15:51:39.802397 32555 ts_tablet_manager.cc:1397] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:51:39.803793 32557 tablet_bootstrap.cc:492] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f: No bootstrap required, opened a new log
I20260808 15:51:39.803987 32557 ts_tablet_manager.cc:1397] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:51:39.806250 32556 raft_consensus.cc:348] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.806542 32556 raft_consensus.cc:374] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:39.806250 32555 raft_consensus.cc:348] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.806597 32556 raft_consensus.cc:729] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df1842515374833b6b9d885bd0af73a, State: Initialized, Role: FOLLOWER
I20260808 15:51:39.806656 32555 raft_consensus.cc:374] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:39.806704 32555 raft_consensus.cc:729] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df2a18d75144994958759b04141cc12, State: Initialized, Role: FOLLOWER
I20260808 15:51:39.806649 32557 raft_consensus.cc:348] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.806913 32557 raft_consensus.cc:374] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:39.806969 32557 raft_consensus.cc:729] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b78401091514575b522a4340f18103f, State: Initialized, Role: FOLLOWER
I20260808 15:51:39.807117 32555 consensus_queue.cc:260] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.807117 32556 consensus_queue.cc:260] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.807407 32557 consensus_queue.cc:260] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.808133 32555 ts_tablet_manager.cc:1428] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:51:39.808156 32156 heartbeater.cc:503] Master 127.30.76.126:34789 was elected leader, sending a full tablet report...
I20260808 15:51:39.808194 32525 heartbeater.cc:503] Master 127.30.76.126:34789 was elected leader, sending a full tablet report...
I20260808 15:51:39.808362 32557 ts_tablet_manager.cc:1428] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 15:51:39.808684 32348 heartbeater.cc:503] Master 127.30.76.126:34789 was elected leader, sending a full tablet report...
I20260808 15:51:39.809304 32556 ts_tablet_manager.cc:1428] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260808 15:51:39.834339 32562 raft_consensus.cc:482] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:39.834561 32562 raft_consensus.cc:504] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.835753 32562 leader_election.cc:288] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6df2a18d75144994958759b04141cc12 (127.30.76.65:34883), 3b78401091514575b522a4340f18103f (127.30.76.66:45527)
I20260808 15:51:39.840201 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.835973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:34883 (local address 127.30.76.67:39341)
0808 15:51:39.836114 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.836132 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.836253 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.836727 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.836733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.836758 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:39.836897 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.836903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.838004 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.838007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.839045 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.839055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.839771 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.839775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.839892 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.839909 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.840008 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.840071 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":43}
I20260808 15:51:39.840189 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.836423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:60515 (local address 127.30.76.66:45527)
0808 15:51:39.836569 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.836574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.836778 (+ 204us) server_negotiation.cc:407] Connection header received
0808 15:51:39.836854 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.836856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.836894 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.836949 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:39.837214 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.837921 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.839204 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.839617 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.839755 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.839812 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.839979 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.840059 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:51:39.840209 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.836273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:45527 (local address 127.30.76.67:60515)
0808 15:51:39.836657 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:39.836688 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:39.836777 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:39.836985 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:39.836989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:39.837011 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:39.837114 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.837118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.838025 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.838029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.839057 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:39.839067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.839744 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.839748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:39.839846 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.839869 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:51:39.839940 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 15:51:39.840018 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":27,"thread_start_us":145,"threads_started":1}
I20260808 15:51:39.840225 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.836088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:39341 (local address 127.30.76.65:34883)
0808 15:51:39.836244 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.836249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.836266 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:51:39.836542 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.836549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.836606 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.836684 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:39.837033 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.837835 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.839177 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.839636 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.839762 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.839813 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.840034 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.840110 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:51:39.840943 32285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6ca24a65c14df5af97ad6bde917977" candidate_uuid: "8df1842515374833b6b9d885bd0af73a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b78401091514575b522a4340f18103f" is_pre_election: true
I20260808 15:51:39.840946 32092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6ca24a65c14df5af97ad6bde917977" candidate_uuid: "8df1842515374833b6b9d885bd0af73a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df2a18d75144994958759b04141cc12" is_pre_election: true
I20260808 15:51:39.841236 32285 raft_consensus.cc:2454] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df1842515374833b6b9d885bd0af73a in term 0.
I20260808 15:51:39.841252 32092 raft_consensus.cc:2454] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df1842515374833b6b9d885bd0af73a in term 0.
I20260808 15:51:39.841707 32382 leader_election.cc:302] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df2a18d75144994958759b04141cc12, 8df1842515374833b6b9d885bd0af73a; no voters:
I20260808 15:51:39.841966 32562 raft_consensus.cc:2781] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:39.842041 32562 raft_consensus.cc:482] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:39.842070 32562 raft_consensus.cc:3037] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:39.843086 32562 raft_consensus.cc:504] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.843412 32562 leader_election.cc:288] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [CANDIDATE]: Term 1 election: Requested vote from peers 6df2a18d75144994958759b04141cc12 (127.30.76.65:34883), 3b78401091514575b522a4340f18103f (127.30.76.66:45527)
I20260808 15:51:39.843773 32092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6ca24a65c14df5af97ad6bde917977" candidate_uuid: "8df1842515374833b6b9d885bd0af73a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df2a18d75144994958759b04141cc12"
I20260808 15:51:39.843858 32285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6ca24a65c14df5af97ad6bde917977" candidate_uuid: "8df1842515374833b6b9d885bd0af73a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b78401091514575b522a4340f18103f"
I20260808 15:51:39.843890 32092 raft_consensus.cc:3037] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:39.843977 32285 raft_consensus.cc:3037] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:39.844997 32092 raft_consensus.cc:2454] T 4d6ca24a65c14df5af97ad6bde917977 P 6df2a18d75144994958759b04141cc12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df1842515374833b6b9d885bd0af73a in term 1.
I20260808 15:51:39.845321 32382 leader_election.cc:302] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df2a18d75144994958759b04141cc12, 8df1842515374833b6b9d885bd0af73a; no voters:
I20260808 15:51:39.845484 32562 raft_consensus.cc:2781] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:39.845587 32285 raft_consensus.cc:2454] T 4d6ca24a65c14df5af97ad6bde917977 P 3b78401091514575b522a4340f18103f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df1842515374833b6b9d885bd0af73a in term 1.
I20260808 15:51:39.845731 32562 raft_consensus.cc:686] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [term 1 LEADER]: Becoming Leader. State: Replica: 8df1842515374833b6b9d885bd0af73a, State: Running, Role: LEADER
I20260808 15:51:39.846066 32562 consensus_queue.cc:237] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } }
I20260808 15:51:39.849004 31874 catalog_manager.cc:5495] T 4d6ca24a65c14df5af97ad6bde917977 P 8df1842515374833b6b9d885bd0af73a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df1842515374833b6b9d885bd0af73a (127.30.76.67). New cstate: current_term: 1 leader_uuid: "8df1842515374833b6b9d885bd0af73a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df1842515374833b6b9d885bd0af73a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 34535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6df2a18d75144994958759b04141cc12" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b78401091514575b522a4340f18103f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 45527 } health_report { overall_health: UNKNOWN } } }
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:39.864313 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.855193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.30.76.65:34883)
0808 15:51:39.855315 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.855336 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.855573 (+ 237us) server_negotiation.cc:407] Connection header received
0808 15:51:39.855743 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.855746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.855798 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.855895 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.856562 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.857045 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.857970 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.858197 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.858304 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.860764 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.860774 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.860787 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.860809 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.862312 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.862848 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.862853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.862854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.862981 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.863305 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.863308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.863310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.863637 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.863847 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.864005 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.864139 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:39.891918 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.883738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.30.76.65:34883)
0808 15:51:39.883860 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.883866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.884028 (+ 162us) server_negotiation.cc:407] Connection header received
0808 15:51:39.884188 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.884191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.884241 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.884326 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.884979 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.885488 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.886305 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.886532 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.886638 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.888995 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.888999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.889001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.889031 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.890214 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.890623 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.890627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.890628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.890707 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.891007 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.891010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.891012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.891179 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.891319 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.891636 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.891754 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:51:39 dist-test-slave-sb2z krb5kdc[31809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:39.917224 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:39.909193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.30.76.65:34883)
0808 15:51:39.909343 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:39.909347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:39.909479 (+ 132us) server_negotiation.cc:407] Connection header received
0808 15:51:39.909640 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:39.909643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:39.909682 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:39.909765 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:39.910398 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.910875 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.911792 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:39.911997 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:39.912101 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:39.914291 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:39.914294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:39.914296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:39.914317 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:39.915757 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.916106 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.916109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.916110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.916180 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:39.916393 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:39.916395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:39.916396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:39.916538 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:39.916642 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:39.916962 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 15:51:39.917076 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 15:51:39.919021 32036 tablet_service.cc:2908] Rejecting scan request for tablet 4d6ca24a65c14df5af97ad6bde917977: Uninitialized: clean time has not yet been initialized
I20260808 15:51:39.921162 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31963
I20260808 15:51:39.930796 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 32166
I20260808 15:51:39.939651 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 32354
I20260808 15:51:39.950868 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31827
2026-08-08T15:51:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2498 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:40 dist-test-slave-sb2z krb5kdc[32585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:40 dist-test-slave-sb2z krb5kdc[32585](info): set up 2 sockets
Aug 08 15:51:40 dist-test-slave-sb2z krb5kdc[32585](info): commencing operation
krb5kdc: starting...
W20260808 15:51:41.163106 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:41.288261 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:37415
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:37415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:41.402320 32604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:41.402609 32604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:41.402683 32604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:41.406687 32604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:41.406769 32604 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 15:51:41.406804 32604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:41.406847 32604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:41.406888 32604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:41.406924 32604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:41.412191 32604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:37415
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:37415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:41.413228 32604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:41.414294 32604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:41.422986 32614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:41.423040 32604 server_base.cc:1034] running on GCE node
W20260808 15:51:41.422982 32613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:41.422982 32616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:41.423900 32604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:41.424997 32604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:41.426193 32604 hybrid_clock.cc:648] HybridClock initialized: now 1786204301426160 us; error 45 us; skew 500 ppm
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:41.431646 32604 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:41.432555 32604 webserver.cc:460] Webserver started at http://127.30.76.126:46859/ using document root <none> and password file <none>
I20260808 15:51:41.433123 32604 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:41.433218 32604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:41.433451 32604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:41.435469 32604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "28774ba9f63d4cd7b00012da0bb64c37"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "d207af7494c475fd6a1ffe9285221a69"
server_key_iv: "b9cd050d0a6f2248e797a095e4822bf9"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.436085 32604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "28774ba9f63d4cd7b00012da0bb64c37"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "d207af7494c475fd6a1ffe9285221a69"
server_key_iv: "b9cd050d0a6f2248e797a095e4822bf9"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.439735 32604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:41.442108 32625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:41.443389 32604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:41.443620 32604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "28774ba9f63d4cd7b00012da0bb64c37"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "d207af7494c475fd6a1ffe9285221a69"
server_key_iv: "b9cd050d0a6f2248e797a095e4822bf9"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.443754 32604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:41.456965 32604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:41.459803 32604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:41.460001 32604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:41.468289 32604 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:37415
I20260808 15:51:41.468307 32714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:37415 every 8 connection(s)
I20260808 15:51:41.469197 32604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:41.471977 32715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:41.474957 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 32604
I20260808 15:51:41.475076 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:41.475364 31025 external_mini_cluster.cc:1428] Setting key f82d855ebeee5fd74035d4b8af083043
I20260808 15:51:41.477902 32715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: Bootstrap starting.
I20260808 15:51:41.480113 32715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:41.480849 32715 log.cc:824] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:41.483052 32715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: No bootstrap required, opened a new log
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:41.487465 32715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28774ba9f63d4cd7b00012da0bb64c37" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37415 } }
I20260808 15:51:41.487768 32715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:41.487869 32715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28774ba9f63d4cd7b00012da0bb64c37, State: Initialized, Role: FOLLOWER
I20260808 15:51:41.488360 32715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28774ba9f63d4cd7b00012da0bb64c37" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37415 } }
I20260808 15:51:41.488520 32715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:41.488605 32715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:41.488726 32715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:41.489841 32715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28774ba9f63d4cd7b00012da0bb64c37" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37415 } }
I20260808 15:51:41.490181 32715 leader_election.cc:302] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28774ba9f63d4cd7b00012da0bb64c37; no voters:
I20260808 15:51:41.490253 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.476811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.30.76.126:37415)
0808 15:51:41.477245 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:41.477260 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:41.477293 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:51:41.477948 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:41.477997 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:41.478359 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:41.478692 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:41.479479 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.480563 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.481631 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.482289 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.482460 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.485086 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:41.485093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:41.485108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:41.485148 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:41.487045 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.487473 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.487481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.487489 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.487648 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.487999 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.488002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.488004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.488487 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:41.488710 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:41.489001 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 15:51:41.489306 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":132,"threads_started":1}
I20260808 15:51:41.490509 32715 leader_election.cc:288] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:41.490617 32722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:41.490847 32722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [term 1 LEADER]: Becoming Leader. State: Replica: 28774ba9f63d4cd7b00012da0bb64c37, State: Running, Role: LEADER
I20260808 15:51:41.491142 32722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28774ba9f63d4cd7b00012da0bb64c37" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37415 } }
I20260808 15:51:41.491751 32715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:41.492322 32722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28774ba9f63d4cd7b00012da0bb64c37. Latest consensus state: current_term: 1 leader_uuid: "28774ba9f63d4cd7b00012da0bb64c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28774ba9f63d4cd7b00012da0bb64c37" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37415 } } }
I20260808 15:51:41.492439 32722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:41.492899 32723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28774ba9f63d4cd7b00012da0bb64c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28774ba9f63d4cd7b00012da0bb64c37" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37415 } } }
I20260808 15:51:41.493067 32723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:41.493105 32730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:41.496234 32730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:41.501394 32730 catalog_manager.cc:1347] Generated new cluster ID: 34f26f32a90b4f15a4e25104d292563a
I20260808 15:51:41.501458 32730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:41.516690 32730 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:41.517562 32730 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:41.521435 32730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: Generated new TSK 0
I20260808 15:51:41.522069 32730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:41.555141 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:37415
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:41.672876 32750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:41.673141 32750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:41.673187 32750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:41.677332 32750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:41.677382 32750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:41.677419 32750 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:51:41.677452 32750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:41.682544 32750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:41.683568 32750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:41.684574 32750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:41.694123 32756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:41.694114 32760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:41.694116 32755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:41.694429 32750 server_base.cc:1034] running on GCE node
I20260808 15:51:41.694788 32750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:41.695370 32750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:41.696534 32750 hybrid_clock.cc:648] HybridClock initialized: now 1786204301696527 us; error 29 us; skew 500 ppm
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:41.699180 32750 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:41.700049 32750 webserver.cc:460] Webserver started at http://127.30.76.65:41711/ using document root <none> and password file <none>
I20260808 15:51:41.700572 32750 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:41.700670 32750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:41.700906 32750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:41.702826 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "06dc8d6d03304f3091f6e339cb58e647"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "d89c1760c0c4f4a3e6f9a7fd8fa22d6e"
server_key_iv: "bb124dcb9055ce44ecd944bda0fb49c2"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.703392 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "06dc8d6d03304f3091f6e339cb58e647"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "d89c1760c0c4f4a3e6f9a7fd8fa22d6e"
server_key_iv: "bb124dcb9055ce44ecd944bda0fb49c2"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.706990 32750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:51:41.709337 301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:41.710476 32750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:41.710610 32750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "06dc8d6d03304f3091f6e339cb58e647"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "d89c1760c0c4f4a3e6f9a7fd8fa22d6e"
server_key_iv: "bb124dcb9055ce44ecd944bda0fb49c2"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.710731 32750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:41.720325 32750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:41.722954 32750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:41.723150 32750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:41.724566 32750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:41.724658 32750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:41.724754 32750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:41.724807 32750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:41.736604 32750 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:35837
I20260808 15:51:41.736931 459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:35837 every 8 connection(s)
I20260808 15:51:41.737529 32750 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:41.739485 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.725463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39377 (local address 127.30.76.126:37415)
0808 15:51:41.725595 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:41.725599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:41.726570 (+ 971us) server_negotiation.cc:407] Connection header received
0808 15:51:41.727641 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:41.727647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:41.727722 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:41.727821 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:41.729124 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.729762 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.730968 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.731372 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.731485 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.735413 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:41.735418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:41.735424 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:41.735459 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:41.737286 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.737857 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.737861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.737863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.737979 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.738386 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.738389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.738391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.738584 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:41.738699 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:41.739176 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 15:51:41.739310 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:51:41.740202 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.725951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.30.76.65:39377)
0808 15:51:41.726424 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:41.726464 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:41.727362 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:41.727967 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:41.727974 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:41.728360 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:41.728934 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.728957 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.729896 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.729899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.730832 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.730842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.731558 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.731564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.731732 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.732309 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:41.732333 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:41.735232 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:41.737416 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.737421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.737435 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.737754 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.738115 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.738118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.738120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.738285 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.738698 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:41.738704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:41.738935 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 15:51:41.739122 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 15:51:41.739423 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":136,"threads_started":1}
I20260808 15:51:41.741477 460 heartbeater.cc:347] Connected to a master server at 127.30.76.126:37415
I20260808 15:51:41.741561 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 32750
I20260808 15:51:41.741658 460 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:41.741715 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:41.741978 31025 external_mini_cluster.cc:1428] Setting key f2b63d4aeaeede89ccd38dd7a5880744
I20260808 15:51:41.742216 460 heartbeater.cc:511] Master 127.30.76.126:37415 requested a full tablet report, sending...
I20260808 15:51:41.743947 32651 ts_manager.cc:194] Registered new tserver with Master: 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837)
I20260808 15:51:41.745550 32651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:39377
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:51:41.754115 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.743916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:43369 (local address 127.30.76.126:37415)
0808 15:51:41.746180 (+ 2264us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:41.746187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:41.746204 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:51:41.746238 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:41.746241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:41.746281 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:41.746370 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:41.747009 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.747779 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.749242 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.749449 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.749528 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.750861 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:41.750864 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:41.750866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:41.750889 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:41.752363 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.752877 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.752879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.752880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.752954 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.753258 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.753260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.753261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.753419 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:41.753489 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:41.753822 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 15:51:41.753914 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2174}
I20260808 15:51:41.754158 471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.743869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.30.76.65:43369)
0808 15:51:41.744172 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:41.744185 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:41.744264 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:41.746414 (+ 2150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:41.746418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:41.746701 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:41.746856 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.746867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.747950 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.747957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.749089 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.749102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.749571 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.749578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.749696 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.750193 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:41.750215 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:41.750717 (+ 502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:41.752478 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.752486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.752490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.752776 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.753043 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.753049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.753052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.753161 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.753518 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:41.753521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:41.753638 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 15:51:41.753769 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 15:51:41.753926 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:41.771924 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:37415
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:41.886175 478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:41.886406 478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:41.886448 478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:41.890481 478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:41.890532 478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:41.890581 478 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:51:41.890676 478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:41.896078 478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:41.897022 478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:41.898067 478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:41.907366 492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:41.907395 487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:41.907401 486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:41.907562 478 server_base.cc:1034] running on GCE node
I20260808 15:51:41.908134 478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:41.908738 478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:41.909930 478 hybrid_clock.cc:648] HybridClock initialized: now 1786204301909899 us; error 42 us; skew 500 ppm
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:41.912773 478 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:41.913638 478 webserver.cc:460] Webserver started at http://127.30.76.66:40839/ using document root <none> and password file <none>
I20260808 15:51:41.914158 478 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:41.914215 478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:41.914417 478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:41.916405 478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "a426c8d02e0c489f8f2dc4ec934d321e"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "6e910abb1261d733d5c1f205954de0f4"
server_key_iv: "3d889db8db3f2a63b3c601c049a0e479"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.916911 478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "a426c8d02e0c489f8f2dc4ec934d321e"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "6e910abb1261d733d5c1f205954de0f4"
server_key_iv: "3d889db8db3f2a63b3c601c049a0e479"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.920473 478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:41.922717 500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:41.923910 478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:41.924081 478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "a426c8d02e0c489f8f2dc4ec934d321e"
format_stamp: "Formatted at 2026-08-08 15:51:41 on dist-test-slave-sb2z"
server_key: "6e910abb1261d733d5c1f205954de0f4"
server_key_iv: "3d889db8db3f2a63b3c601c049a0e479"
server_key_version: "encryptionkey@0"
I20260808 15:51:41.924223 478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:41.937523 478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:41.940573 478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:41.940719 478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:41.942178 478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:41.942225 478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:41.942286 478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:41.942318 478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:51:41 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:41.953665 478 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:40821
I20260808 15:51:41.954512 478 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:41.955320 661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:40821 every 8 connection(s)
I20260808 15:51:41.956266 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.943076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:47411 (local address 127.30.76.126:37415)
0808 15:51:41.943205 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:41.943209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:41.944195 (+ 986us) server_negotiation.cc:407] Connection header received
0808 15:51:41.945218 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:41.945225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:41.945294 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:41.945360 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:41.947142 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.948058 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.949725 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.950008 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.950122 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.952619 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:41.952624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:41.952626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:41.952660 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:41.954144 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.954630 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.954637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.954640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.954724 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.955110 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.955113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.955114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.955340 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:41.955451 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:41.955978 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 15:51:41.956096 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:51:41.956949 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.943581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.30.76.66:47411)
0808 15:51:41.944045 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:41.944084 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:41.944950 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:41.945816 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:41.945825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:41.946318 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:41.946922 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.946947 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.948214 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.948217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.949602 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.949612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.950171 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.950175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.950360 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.950890 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:41.950905 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:41.952438 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:41.954259 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.954265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.954278 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.954523 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.954825 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.954830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.954833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.954997 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.955448 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:41.955455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:41.955743 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 15:51:41.955920 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 15:51:41.956223 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260808 15:51:41.958065 662 heartbeater.cc:347] Connected to a master server at 127.30.76.126:37415
I20260808 15:51:41.958197 662 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:41.958715 662 heartbeater.cc:511] Master 127.30.76.126:37415 requested a full tablet report, sending...
I20260808 15:51:41.958724 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 478
I20260808 15:51:41.958855 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:41.959192 31025 external_mini_cluster.cc:1428] Setting key 44bb2091384bfd19ffebd82fbf67cade
I20260808 15:51:41.959803 32650 ts_manager.cc:194] Registered new tserver with Master: a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66:40821)
I20260808 15:51:41.960580 32650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:47411
I20260808 15:51:41.968031 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.960434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.30.76.66:37947)
0808 15:51:41.960822 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:41.960837 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:41.960981 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:41.961251 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:41.961254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:41.961462 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:41.961566 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.961572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.962595 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.962603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.963121 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:41.963137 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.963741 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.963744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:41.963809 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.964266 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:41.964277 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:41.964562 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:41.966477 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.966480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.966481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.966663 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.966982 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:41.966984 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:41.966985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:41.967057 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:41.967648 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:41.967650 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:41.967722 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 15:51:41.967805 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:51:41.967899 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":91,"threads_started":1}
I20260808 15:51:41.968207 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:41.960415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:37947 (local address 127.30.76.126:37415)
0808 15:51:41.960528 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:41.960532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:41.960899 (+ 367us) server_negotiation.cc:407] Connection header received
0808 15:51:41.961090 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:41.961097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:41.961153 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:41.961236 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:41.961698 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.962454 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.963244 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:41.963634 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:41.963746 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:41.964713 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:41.964720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:41.964726 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:41.964757 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:41.966374 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.966768 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.966774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.966779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.966878 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:41.967153 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:41.967159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:41.967163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:41.967474 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:41.967655 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:41.967903 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 15:51:41.968024 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:41.986552 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:37415
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:42.100248 672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:42.100477 672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:42.100523 672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:42.104533 672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:42.104590 672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:42.104662 672 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:51:42.104743 672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:42.109823 672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:42.110767 672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:42.111761 672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:42.121193 678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:42.121170 685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:42.121204 683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:42.121573 672 server_base.cc:1034] running on GCE node
I20260808 15:51:42.121950 672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:42.122499 672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:42.123615 672 hybrid_clock.cc:648] HybridClock initialized: now 1786204302123596 us; error 30 us; skew 500 ppm
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204302, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:42.126401 672 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:42.127164 672 webserver.cc:460] Webserver started at http://127.30.76.67:33799/ using document root <none> and password file <none>
I20260808 15:51:42.127772 672 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:42.127836 672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:42.128008 672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:42.129964 672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "e8108b3987c14bcbb2875833102e8f35"
format_stamp: "Formatted at 2026-08-08 15:51:42 on dist-test-slave-sb2z"
server_key: "d8608f77b12c6e024155a8948c37d678"
server_key_iv: "c3f7d2b0f5199302454ada42ce0aed51"
server_key_version: "encryptionkey@0"
I20260808 15:51:42.130470 672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "e8108b3987c14bcbb2875833102e8f35"
format_stamp: "Formatted at 2026-08-08 15:51:42 on dist-test-slave-sb2z"
server_key: "d8608f77b12c6e024155a8948c37d678"
server_key_iv: "c3f7d2b0f5199302454ada42ce0aed51"
server_key_version: "encryptionkey@0"
I20260808 15:51:42.134017 672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 15:51:42.136276 699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:42.137491 672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:51:42.137622 672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "e8108b3987c14bcbb2875833102e8f35"
format_stamp: "Formatted at 2026-08-08 15:51:42 on dist-test-slave-sb2z"
server_key: "d8608f77b12c6e024155a8948c37d678"
server_key_iv: "c3f7d2b0f5199302454ada42ce0aed51"
server_key_version: "encryptionkey@0"
I20260808 15:51:42.137724 672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:42.150640 672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:42.153872 672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:42.154021 672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:42.155447 672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:42.155503 672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:42.155658 672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:42.155701 672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204302, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:42.167169 672 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44679
I20260808 15:51:42.167464 845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44679 every 8 connection(s)
I20260808 15:51:42.168339 672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:42.170343 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.156360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:33935 (local address 127.30.76.126:37415)
0808 15:51:42.156492 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.156496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.157502 (+ 1006us) server_negotiation.cc:407] Connection header received
0808 15:51:42.158534 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.158538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.158603 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.158701 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:42.160286 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.161163 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.162896 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.163310 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.163447 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.166236 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:42.166240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:42.166243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:42.166271 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:42.168142 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.168670 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.168676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.168680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.168783 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.169214 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.169219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.169221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.169430 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:42.169546 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.170027 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.170178 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:51:42.171062 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.156880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.30.76.67:33935)
0808 15:51:42.157360 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.157400 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.158204 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.158960 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.158967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.159513 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:42.160142 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.160162 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.161275 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.161279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.162766 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.162780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.163502 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.163506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.163736 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.164331 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:42.164346 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:42.166058 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:42.168247 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:42.168252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:42.168262 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:42.168565 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:42.168901 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:42.168904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:42.168906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:42.169100 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:42.169555 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:42.169564 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:42.169784 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.169975 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.170267 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260808 15:51:42.172190 846 heartbeater.cc:347] Connected to a master server at 127.30.76.126:37415
I20260808 15:51:42.172326 846 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:42.172850 846 heartbeater.cc:511] Master 127.30.76.126:37415 requested a full tablet report, sending...
I20260808 15:51:42.173195 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 672
I20260808 15:51:42.173323 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:42.173583 31025 external_mini_cluster.cc:1428] Setting key f24aa55d9b0644286b7f82bea61dfc52
I20260808 15:51:42.174088 32651 ts_manager.cc:194] Registered new tserver with Master: e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679)
I20260808 15:51:42.174856 32651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:33935
I20260808 15:51:42.176663 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:51:42.182051 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.174509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:34747 (local address 127.30.76.126:37415)
0808 15:51:42.174610 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.174614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.174913 (+ 299us) server_negotiation.cc:407] Connection header received
0808 15:51:42.175050 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.175053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.175098 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.175181 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:42.175797 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.176483 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.177405 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.177797 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.177918 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.179116 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:42.179120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:42.179122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:42.179160 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:42.180479 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.180898 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.180901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.180903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.180992 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.181275 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.181279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.181280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.181444 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:42.181550 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.181815 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.181922 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 15:51:42.182065 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.174439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.30.76.67:34747)
0808 15:51:42.174823 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.174835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.174916 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.175214 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.175216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.175421 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:42.175566 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.175575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.176619 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.176623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.177284 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.177293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.177918 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.177923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.178043 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.178552 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:42.178569 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:42.178987 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:42.180588 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:42.180593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:42.180595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:42.180796 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:42.181087 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:42.181091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:42.181092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:42.181180 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:42.181542 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:42.181545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:42.181647 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.181757 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.181888 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":209,"threads_started":1}
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204302, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204302, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:42.204337 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.196448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.30.76.126:37415)
0808 15:51:42.196579 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.196582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.196594 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:51:42.196627 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.196630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.196664 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.196734 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:42.197405 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.197866 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.198615 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.198802 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.198893 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.201187 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:42.201191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:42.201193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:42.201214 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:42.202590 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.203022 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.203030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.203032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.203104 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.203402 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.203406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.203408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.203653 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:42.203796 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.204045 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.204169 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:51:42.207072 32650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:51:42.210206 32650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:42.224671 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.218578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.30.76.67:44679)
0808 15:51:42.219401 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.219414 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.219439 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:51:42.219545 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.219554 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.219780 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.220312 (+ 532us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.220477 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.221501 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.223171 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.223906 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.224032 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.224213 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.224417 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.224506 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":141,"threads_started":1}
I20260808 15:51:42.224663 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.218450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44679 (local address 127.0.0.1:56778)
0808 15:51:42.218918 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.218966 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.219095 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.219965 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.219971 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.219992 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.220151 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.220162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.221624 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.221630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.223024 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.223035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.224033 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.224037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.224208 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.224258 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.224387 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.224507 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":128,"thread_start_us":60,"threads_started":1}
I20260808 15:51:42.224956 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.219137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40821 (local address 127.0.0.1:36510)
0808 15:51:42.219585 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.219598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.219703 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.220786 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.220788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.220807 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.220885 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.220890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.222260 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.222263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.223353 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.223362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.224576 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.224578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.224649 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.224666 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.224759 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.224829 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":101,"threads_started":1}
I20260808 15:51:42.225010 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.219602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.30.76.66:40821)
0808 15:51:42.219943 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.219956 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.219978 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:51:42.220050 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.220055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.220230 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.220644 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.221036 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.222134 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.223486 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.224305 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.224411 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.224524 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.224791 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.224875 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":123,"threads_started":1}
I20260808 15:51:42.225332 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.218526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:35837 (local address 127.0.0.1:50250)
0808 15:51:42.219258 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.219272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.219356 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.220423 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.220426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.220446 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.220533 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.220542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.221728 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.221731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.223022 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.223032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.224073 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.224076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.225050 (+ 974us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.225073 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.225139 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.225207 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":887,"thread_start_us":112,"threads_started":1}
I20260808 15:51:42.225740 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.219094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.30.76.65:35837)
0808 15:51:42.219943 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.219958 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.219978 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:51:42.220051 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.220055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.220230 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.220433 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.220661 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.221531 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.223147 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.223926 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.224196 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.224359 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.225505 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.225595 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":117,"threads_started":1}
I20260808 15:51:42.228044 766 tablet_service.cc:1467] Processing CreateTablet for tablet df7a9031ca6d487da08ddbd249835327 (DEFAULT_TABLE table=test-table [id=281e6dfbe95248bd851bc12ce5827a8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:42.228124 369 tablet_service.cc:1467] Processing CreateTablet for tablet df7a9031ca6d487da08ddbd249835327 (DEFAULT_TABLE table=test-table [id=281e6dfbe95248bd851bc12ce5827a8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:42.229035 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7a9031ca6d487da08ddbd249835327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.229027 369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7a9031ca6d487da08ddbd249835327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.235704 876 tablet_bootstrap.cc:492] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Bootstrap starting.
I20260808 15:51:42.235834 570 tablet_service.cc:1467] Processing CreateTablet for tablet df7a9031ca6d487da08ddbd249835327 (DEFAULT_TABLE table=test-table [id=281e6dfbe95248bd851bc12ce5827a8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:42.236871 570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7a9031ca6d487da08ddbd249835327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.236920 877 tablet_bootstrap.cc:492] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Bootstrap starting.
I20260808 15:51:42.238149 876 tablet_bootstrap.cc:654] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.238981 877 tablet_bootstrap.cc:654] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.239046 876 log.cc:824] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:42.239805 877 log.cc:824] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:42.241269 876 tablet_bootstrap.cc:492] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: No bootstrap required, opened a new log
I20260808 15:51:42.241463 876 ts_tablet_manager.cc:1397] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:51:42.241807 877 tablet_bootstrap.cc:492] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: No bootstrap required, opened a new log
I20260808 15:51:42.241990 877 ts_tablet_manager.cc:1397] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:51:42.244314 884 tablet_bootstrap.cc:492] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Bootstrap starting.
I20260808 15:51:42.245437 876 raft_consensus.cc:348] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.245710 876 raft_consensus.cc:374] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.245800 876 raft_consensus.cc:729] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8108b3987c14bcbb2875833102e8f35, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.245682 877 raft_consensus.cc:348] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.245951 877 raft_consensus.cc:374] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.246011 877 raft_consensus.cc:729] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dc8d6d03304f3091f6e339cb58e647, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.246250 876 consensus_queue.cc:260] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.246399 877 consensus_queue.cc:260] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.246519 884 tablet_bootstrap.cc:654] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.247007 460 heartbeater.cc:503] Master 127.30.76.126:37415 was elected leader, sending a full tablet report...
I20260808 15:51:42.247262 877 ts_tablet_manager.cc:1428] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 15:51:42.247260 876 ts_tablet_manager.cc:1428] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:51:42.247598 846 heartbeater.cc:503] Master 127.30.76.126:37415 was elected leader, sending a full tablet report...
I20260808 15:51:42.250052 884 log.cc:824] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:42.251991 884 tablet_bootstrap.cc:492] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: No bootstrap required, opened a new log
I20260808 15:51:42.252142 884 ts_tablet_manager.cc:1397] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 15:51:42.254681 884 raft_consensus.cc:348] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.254863 884 raft_consensus.cc:374] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.254902 884 raft_consensus.cc:729] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a426c8d02e0c489f8f2dc4ec934d321e, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.255265 884 consensus_queue.cc:260] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.255856 662 heartbeater.cc:503] Master 127.30.76.126:37415 was elected leader, sending a full tablet report...
I20260808 15:51:42.256088 884 ts_tablet_manager.cc:1428] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 15:51:42.335567 888 raft_consensus.cc:482] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:42.335768 888 raft_consensus.cc:504] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.336843 888 leader_election.cc:288] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679), a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66:40821)
I20260808 15:51:42.341048 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.337299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:41101 (local address 127.30.76.66:40821)
0808 15:51:42.337418 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.337423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.337771 (+ 348us) server_negotiation.cc:407] Connection header received
0808 15:51:42.337965 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.337969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.338019 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.338105 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.338427 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.338956 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.340040 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.340435 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.340529 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.340569 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.340822 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.340915 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:51:42.341104 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.337090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40821 (local address 127.30.76.65:41101)
0808 15:51:42.337664 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.337681 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.337788 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.338117 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.338120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.338139 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.338277 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.338284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.339075 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.339078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.339918 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.339927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.340567 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.340571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.340684 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.340711 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.340788 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.340849 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":320,"thread_start_us":87,"threads_started":1}
I20260808 15:51:42.341120 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.337082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44679 (local address 127.30.76.65:47025)
0808 15:51:42.337399 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.337416 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.337540 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.337965 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.337968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.337984 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.338105 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.338114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.339006 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.339011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.339931 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.339941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.340703 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.340706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.340812 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.340837 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.340911 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.340985 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":18,"thread_start_us":115,"threads_started":1}
I20260808 15:51:42.341167 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.337250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:47025 (local address 127.30.76.67:44679)
0808 15:51:42.337390 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.337395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.337511 (+ 116us) server_negotiation.cc:407] Connection header received
0808 15:51:42.337691 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.337695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.337753 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.337810 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.338298 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.338878 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.340084 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.340584 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.340699 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.340747 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.340971 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.341038 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:51:42.341879 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7a9031ca6d487da08ddbd249835327" candidate_uuid: "06dc8d6d03304f3091f6e339cb58e647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8108b3987c14bcbb2875833102e8f35" is_pre_election: true
I20260808 15:51:42.341879 598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7a9031ca6d487da08ddbd249835327" candidate_uuid: "06dc8d6d03304f3091f6e339cb58e647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" is_pre_election: true
I20260808 15:51:42.342177 598 raft_consensus.cc:2454] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06dc8d6d03304f3091f6e339cb58e647 in term 0.
I20260808 15:51:42.342177 789 raft_consensus.cc:2454] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06dc8d6d03304f3091f6e339cb58e647 in term 0.
I20260808 15:51:42.342656 302 leader_election.cc:302] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dc8d6d03304f3091f6e339cb58e647, a426c8d02e0c489f8f2dc4ec934d321e; no voters:
I20260808 15:51:42.342957 888 raft_consensus.cc:2781] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:42.343044 888 raft_consensus.cc:482] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:42.343092 888 raft_consensus.cc:3037] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.344125 888 raft_consensus.cc:504] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.344430 888 leader_election.cc:288] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [CANDIDATE]: Term 1 election: Requested vote from peers e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679), a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66:40821)
I20260808 15:51:42.344774 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7a9031ca6d487da08ddbd249835327" candidate_uuid: "06dc8d6d03304f3091f6e339cb58e647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8108b3987c14bcbb2875833102e8f35"
I20260808 15:51:42.344774 598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7a9031ca6d487da08ddbd249835327" candidate_uuid: "06dc8d6d03304f3091f6e339cb58e647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a426c8d02e0c489f8f2dc4ec934d321e"
I20260808 15:51:42.344895 598 raft_consensus.cc:3037] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.344895 789 raft_consensus.cc:3037] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.346540 598 raft_consensus.cc:2454] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06dc8d6d03304f3091f6e339cb58e647 in term 1.
I20260808 15:51:42.346540 789 raft_consensus.cc:2454] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06dc8d6d03304f3091f6e339cb58e647 in term 1.
I20260808 15:51:42.346841 302 leader_election.cc:302] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dc8d6d03304f3091f6e339cb58e647, a426c8d02e0c489f8f2dc4ec934d321e; no voters:
I20260808 15:51:42.347023 888 raft_consensus.cc:2781] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:42.347242 888 raft_consensus.cc:686] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 LEADER]: Becoming Leader. State: Replica: 06dc8d6d03304f3091f6e339cb58e647, State: Running, Role: LEADER
I20260808 15:51:42.347604 888 consensus_queue.cc:237] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } }
I20260808 15:51:42.350872 32651 catalog_manager.cc:5495] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65). New cstate: current_term: 1 leader_uuid: "06dc8d6d03304f3091f6e339cb58e647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } health_report { overall_health: UNKNOWN } } }
I20260808 15:51:42.380882 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.376992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.30.76.65:35837)
0808 15:51:42.377160 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.377165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.377182 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:51:42.377401 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.377406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.377476 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.377582 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:42.377874 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.378380 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.379273 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.379492 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.379651 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.380040 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:42.380427 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:42.380555 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.380677 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.380735 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 15:51:42.386264 789 raft_consensus.cc:1261] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Refusing update from remote peer 06dc8d6d03304f3091f6e339cb58e647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:42.386265 598 raft_consensus.cc:1261] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Refusing update from remote peer 06dc8d6d03304f3091f6e339cb58e647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:42.386943 895 consensus_queue.cc:1035] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:42.387121 888 consensus_queue.cc:1035] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:42.394155 900 mvcc.cc:205] Tried to move back new op lower bound from 7316292822567587840 to 7316292822418636800. Current Snapshot: MvccSnapshot[applied={T|T < 7316292822567587840}]
I20260808 15:51:42.396299 905 mvcc.cc:205] Tried to move back new op lower bound from 7316292822567587840 to 7316292822418636800. Current Snapshot: MvccSnapshot[applied={T|T < 7316292822567587840}]
I20260808 15:51:42.397243 904 mvcc.cc:205] Tried to move back new op lower bound from 7316292822567587840 to 7316292822418636800. Current Snapshot: MvccSnapshot[applied={T|T < 7316292822567587840 or (T in {7316292822567587840})}]
I20260808 15:51:42.406417 32651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:42.406590 32651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:42.409118 32651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: Sending DeleteTablet for 3 replicas of tablet df7a9031ca6d487da08ddbd249835327
I20260808 15:51:42.409871 766 tablet_service.cc:1514] Processing DeleteTablet for tablet df7a9031ca6d487da08ddbd249835327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:42 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56778
I20260808 15:51:42.409914 570 tablet_service.cc:1514] Processing DeleteTablet for tablet df7a9031ca6d487da08ddbd249835327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:42 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:36510
I20260808 15:51:42.410475 916 tablet_replica.cc:330] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: stopping tablet replica
I20260808 15:51:42.410501 369 tablet_service.cc:1514] Processing DeleteTablet for tablet df7a9031ca6d487da08ddbd249835327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:42 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:50250
I20260808 15:51:42.410534 32651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:51:42.410830 916 raft_consensus.cc:2229] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 15:51:42.410931 32651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:42.411101 916 raft_consensus.cc:2258] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:42.411123 917 tablet_replica.cc:330] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: stopping tablet replica
I20260808 15:51:42.411383 917 raft_consensus.cc:2229] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:51:42.412289 917 raft_consensus.cc:2258] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:42.412683 916 ts_tablet_manager.cc:1905] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:42.413657 915 tablet_replica.cc:330] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: stopping tablet replica
I20260808 15:51:42.414789 917 ts_tablet_manager.cc:1905] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:42.415060 915 raft_consensus.cc:2229] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:42.415393 915 raft_consensus.cc:2258] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:42.415761 916 ts_tablet_manager.cc:1918] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:51:42.415922 916 log.cc:1196] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/df7a9031ca6d487da08ddbd249835327
I20260808 15:51:42.416303 916 ts_tablet_manager.cc:1939] T df7a9031ca6d487da08ddbd249835327 P e8108b3987c14bcbb2875833102e8f35: Deleting consensus metadata
I20260808 15:51:42.416855 369 tablet_service.cc:1467] Processing CreateTablet for tablet 674693dcf0bd443da231743a081d0d97 (DEFAULT_TABLE table=test-table [id=520f3c4022b44d218f71bbfc8f951994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:42.417143 369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 674693dcf0bd443da231743a081d0d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.417762 766 tablet_service.cc:1467] Processing CreateTablet for tablet 674693dcf0bd443da231743a081d0d97 (DEFAULT_TABLE table=test-table [id=520f3c4022b44d218f71bbfc8f951994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:42.417984 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 674693dcf0bd443da231743a081d0d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.418462 917 ts_tablet_manager.cc:1918] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:51:42.418550 917 log.cc:1196] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/df7a9031ca6d487da08ddbd249835327
I20260808 15:51:42.418514 32629 catalog_manager.cc:4841] TS e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679): tablet df7a9031ca6d487da08ddbd249835327 (table test-table [id=281e6dfbe95248bd851bc12ce5827a8a]) successfully deleted
I20260808 15:51:42.418900 917 ts_tablet_manager.cc:1939] T df7a9031ca6d487da08ddbd249835327 P 06dc8d6d03304f3091f6e339cb58e647: Deleting consensus metadata
I20260808 15:51:42.419945 877 tablet_bootstrap.cc:492] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Bootstrap starting.
I20260808 15:51:42.420122 876 tablet_bootstrap.cc:492] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Bootstrap starting.
I20260808 15:51:42.420264 570 tablet_service.cc:1467] Processing CreateTablet for tablet 674693dcf0bd443da231743a081d0d97 (DEFAULT_TABLE table=test-table [id=520f3c4022b44d218f71bbfc8f951994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:42.420498 570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 674693dcf0bd443da231743a081d0d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.420954 877 tablet_bootstrap.cc:654] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.421021 876 tablet_bootstrap.cc:654] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.421578 32626 catalog_manager.cc:4841] TS 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837): tablet df7a9031ca6d487da08ddbd249835327 (table test-table [id=281e6dfbe95248bd851bc12ce5827a8a]) successfully deleted
I20260808 15:51:42.422741 877 tablet_bootstrap.cc:492] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: No bootstrap required, opened a new log
I20260808 15:51:42.422813 877 ts_tablet_manager.cc:1397] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:51:42.422823 915 ts_tablet_manager.cc:1905] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:42.422873 884 tablet_bootstrap.cc:492] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Bootstrap starting.
I20260808 15:51:42.423290 877 raft_consensus.cc:348] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.423405 877 raft_consensus.cc:374] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.423435 877 raft_consensus.cc:729] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dc8d6d03304f3091f6e339cb58e647, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.423651 877 consensus_queue.cc:260] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.423949 877 ts_tablet_manager.cc:1428] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:42.424342 884 tablet_bootstrap.cc:654] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.425554 915 ts_tablet_manager.cc:1918] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:51:42.425697 915 log.cc:1196] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/df7a9031ca6d487da08ddbd249835327
I20260808 15:51:42.425920 884 tablet_bootstrap.cc:492] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: No bootstrap required, opened a new log
I20260808 15:51:42.425998 884 ts_tablet_manager.cc:1397] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:51:42.426046 915 ts_tablet_manager.cc:1939] T df7a9031ca6d487da08ddbd249835327 P a426c8d02e0c489f8f2dc4ec934d321e: Deleting consensus metadata
I20260808 15:51:42.426421 884 raft_consensus.cc:348] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.426512 884 raft_consensus.cc:374] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.426541 884 raft_consensus.cc:729] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a426c8d02e0c489f8f2dc4ec934d321e, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.426669 884 consensus_queue.cc:260] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.426906 884 ts_tablet_manager.cc:1428] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:42.427680 32626 catalog_manager.cc:4841] TS a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66:40821): tablet df7a9031ca6d487da08ddbd249835327 (table test-table [id=281e6dfbe95248bd851bc12ce5827a8a]) successfully deleted
I20260808 15:51:42.428094 876 tablet_bootstrap.cc:492] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: No bootstrap required, opened a new log
I20260808 15:51:42.428185 876 ts_tablet_manager.cc:1397] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 15:51:42.428599 876 raft_consensus.cc:348] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.428700 876 raft_consensus.cc:374] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.428731 876 raft_consensus.cc:729] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8108b3987c14bcbb2875833102e8f35, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.428854 876 consensus_queue.cc:260] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.429080 876 ts_tablet_manager.cc:1428] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260808 15:51:42.455708 663 tablet.cc:2369] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:51:42.488906 462 tablet.cc:2369] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:51:42.540102 890 raft_consensus.cc:482] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:42.540304 890 raft_consensus.cc:504] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.541882 890 leader_election.cc:288] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837), e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679)
I20260808 15:51:42.546664 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.542381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:40055 (local address 127.30.76.65:35837)
0808 15:51:42.542513 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.542517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.542622 (+ 105us) server_negotiation.cc:407] Connection header received
0808 15:51:42.542875 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.542878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.542930 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.542970 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.543409 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.544270 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.545616 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.546005 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.546145 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.546205 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.546408 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.546497 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:51:42.546718 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.542150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44679 (local address 127.30.76.66:37047)
0808 15:51:42.542695 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.542709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.542778 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.543068 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.543073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.543098 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.543265 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.543270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.544117 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.544119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.545338 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.545348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.546302 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.546308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.546416 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.546438 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.546507 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.546575 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":107,"thread_start_us":88,"threads_started":1}
I20260808 15:51:42.546795 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.542637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:37047 (local address 127.30.76.67:44679)
0808 15:51:42.542779 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.542784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.542805 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:51:42.542867 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.542871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.542924 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.543034 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:42.543401 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.543994 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.545491 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.546185 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.546324 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.546380 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.546557 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.546636 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:51:42.547180 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.542144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:35837 (local address 127.30.76.66:40055)
0808 15:51:42.542519 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.542534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.542738 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.543133 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.543136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.543148 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:42.543249 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.543257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.544406 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.544409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.545497 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.545504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.546131 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.546134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.546237 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.546268 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.546936 (+ 668us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.547015 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":4,"thread_start_us":105,"threads_started":1}
I20260808 15:51:42.547801 395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674693dcf0bd443da231743a081d0d97" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dc8d6d03304f3091f6e339cb58e647" is_pre_election: true
I20260808 15:51:42.547873 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674693dcf0bd443da231743a081d0d97" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8108b3987c14bcbb2875833102e8f35" is_pre_election: true
I20260808 15:51:42.548045 789 raft_consensus.cc:2454] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 0.
I20260808 15:51:42.548079 395 raft_consensus.cc:2454] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 0.
I20260808 15:51:42.548478 503 leader_election.cc:302] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a426c8d02e0c489f8f2dc4ec934d321e, e8108b3987c14bcbb2875833102e8f35; no voters:
I20260808 15:51:42.548774 890 raft_consensus.cc:2781] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:42.548857 890 raft_consensus.cc:482] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:42.548910 890 raft_consensus.cc:3037] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.550365 890 raft_consensus.cc:504] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.550778 890 leader_election.cc:288] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 election: Requested vote from peers 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837), e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679)
I20260808 15:51:42.551092 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674693dcf0bd443da231743a081d0d97" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8108b3987c14bcbb2875833102e8f35"
I20260808 15:51:42.551092 395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674693dcf0bd443da231743a081d0d97" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dc8d6d03304f3091f6e339cb58e647"
I20260808 15:51:42.551220 789 raft_consensus.cc:3037] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.551220 395 raft_consensus.cc:3037] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.552428 395 raft_consensus.cc:2454] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 1.
I20260808 15:51:42.552428 789 raft_consensus.cc:2454] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 1.
I20260808 15:51:42.552750 501 leader_election.cc:302] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dc8d6d03304f3091f6e339cb58e647, a426c8d02e0c489f8f2dc4ec934d321e; no voters:
I20260808 15:51:42.552951 890 raft_consensus.cc:2781] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:42.553188 890 raft_consensus.cc:686] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 LEADER]: Becoming Leader. State: Replica: a426c8d02e0c489f8f2dc4ec934d321e, State: Running, Role: LEADER
I20260808 15:51:42.553627 890 consensus_queue.cc:237] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.555994 32651 catalog_manager.cc:5495] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e reported cstate change: term changed from 0 to 1, leader changed from <none> to a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66). New cstate: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } health_report { overall_health: UNKNOWN } } }
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204301, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:42.617667 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.609240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37415 (local address 127.0.0.1:36688)
0808 15:51:42.609561 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.609574 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.609721 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.609972 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.609974 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.610403 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:42.610493 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.610499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.611274 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.611280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.611906 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.611915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.612369 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.612372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.612487 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.613002 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:42.613014 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:42.614591 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:42.616104 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:42.616112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:42.616117 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:42.616343 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:42.616592 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:42.616594 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:42.616595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:42.616686 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:42.617078 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:42.617084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:42.617190 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.617295 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.617465 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":19,"thread_start_us":119,"threads_started":1}
I20260808 15:51:42.617669 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.609272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.30.76.126:37415)
0808 15:51:42.609443 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.609450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.609663 (+ 213us) server_negotiation.cc:407] Connection header received
0808 15:51:42.609818 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.609821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.609868 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.609953 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:42.610646 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.611164 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.612026 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.612245 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.612355 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.614729 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:42.614733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:42.614735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:42.614758 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:42.616001 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.616432 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.616434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.616435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.616510 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.616763 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.616766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.616767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.616927 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:42.617026 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.617363 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.617517 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":63}
I20260808 15:51:42.621619 32651 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:36688:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 15:51:42.622368 32651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:42.626986 369 tablet_service.cc:1467] Processing CreateTablet for tablet 90d666687d0f441995bf8bf2f45e8a5a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0ecae6d17afa47b58be26181c0a54e4f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:51:42.627010 570 tablet_service.cc:1467] Processing CreateTablet for tablet 90d666687d0f441995bf8bf2f45e8a5a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0ecae6d17afa47b58be26181c0a54e4f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:51:42.627221 766 tablet_service.cc:1467] Processing CreateTablet for tablet 90d666687d0f441995bf8bf2f45e8a5a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0ecae6d17afa47b58be26181c0a54e4f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:51:42.627403 570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d666687d0f441995bf8bf2f45e8a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.627400 369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d666687d0f441995bf8bf2f45e8a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.627589 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d666687d0f441995bf8bf2f45e8a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:42.630218 877 tablet_bootstrap.cc:492] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: Bootstrap starting.
I20260808 15:51:42.630358 884 tablet_bootstrap.cc:492] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: Bootstrap starting.
I20260808 15:51:42.630439 876 tablet_bootstrap.cc:492] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: Bootstrap starting.
I20260808 15:51:42.630946 877 tablet_bootstrap.cc:654] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.631363 884 tablet_bootstrap.cc:654] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.631363 876 tablet_bootstrap.cc:654] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:42.632102 877 tablet_bootstrap.cc:492] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: No bootstrap required, opened a new log
I20260808 15:51:42.632206 877 ts_tablet_manager.cc:1397] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:42.632664 877 raft_consensus.cc:348] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.632757 884 tablet_bootstrap.cc:492] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: No bootstrap required, opened a new log
I20260808 15:51:42.632790 877 raft_consensus.cc:374] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.632808 876 tablet_bootstrap.cc:492] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: No bootstrap required, opened a new log
I20260808 15:51:42.632834 884 ts_tablet_manager.cc:1397] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 15:51:42.632833 877 raft_consensus.cc:729] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dc8d6d03304f3091f6e339cb58e647, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.632874 876 ts_tablet_manager.cc:1397] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 15:51:42.632956 877 consensus_queue.cc:260] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.633270 877 ts_tablet_manager.cc:1428] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:42.633302 884 raft_consensus.cc:348] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.633302 876 raft_consensus.cc:348] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.633412 884 raft_consensus.cc:374] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.633419 876 raft_consensus.cc:374] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:42.633466 884 raft_consensus.cc:729] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a426c8d02e0c489f8f2dc4ec934d321e, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.633466 876 raft_consensus.cc:729] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8108b3987c14bcbb2875833102e8f35, State: Initialized, Role: FOLLOWER
I20260808 15:51:42.633446 912 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.633602 912 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:42.633592 876 consensus_queue.cc:260] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.633592 884 consensus_queue.cc:260] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.633869 890 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.633846 886 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.633997 890 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:42.633997 886 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:42.633850 876 ts_tablet_manager.cc:1428] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 15:51:42.634512 884 ts_tablet_manager.cc:1428] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 15:51:42.642835 890 raft_consensus.cc:482] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:42.642910 890 raft_consensus.cc:504] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.643275 890 leader_election.cc:288] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837), e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679)
I20260808 15:51:42.643625 395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d666687d0f441995bf8bf2f45e8a5a" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dc8d6d03304f3091f6e339cb58e647" is_pre_election: true
I20260808 15:51:42.643651 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d666687d0f441995bf8bf2f45e8a5a" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8108b3987c14bcbb2875833102e8f35" is_pre_election: true
I20260808 15:51:42.643764 395 raft_consensus.cc:2454] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 0.
I20260808 15:51:42.643775 789 raft_consensus.cc:2454] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 0.
I20260808 15:51:42.644027 503 leader_election.cc:302] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a426c8d02e0c489f8f2dc4ec934d321e, e8108b3987c14bcbb2875833102e8f35; no voters:
I20260808 15:51:42.644224 890 raft_consensus.cc:2781] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:42.644289 890 raft_consensus.cc:482] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:42.644328 890 raft_consensus.cc:3037] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.645339 890 raft_consensus.cc:504] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.645661 890 leader_election.cc:288] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 election: Requested vote from peers 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837), e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679)
I20260808 15:51:42.645911 395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d666687d0f441995bf8bf2f45e8a5a" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dc8d6d03304f3091f6e339cb58e647"
I20260808 15:51:42.646026 395 raft_consensus.cc:3037] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.646011 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d666687d0f441995bf8bf2f45e8a5a" candidate_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8108b3987c14bcbb2875833102e8f35"
I20260808 15:51:42.646114 789 raft_consensus.cc:3037] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:42.647147 395 raft_consensus.cc:2454] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 1.
I20260808 15:51:42.647140 789 raft_consensus.cc:2454] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a426c8d02e0c489f8f2dc4ec934d321e in term 1.
I20260808 15:51:42.647398 503 leader_election.cc:302] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a426c8d02e0c489f8f2dc4ec934d321e, e8108b3987c14bcbb2875833102e8f35; no voters:
I20260808 15:51:42.647650 890 raft_consensus.cc:2781] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:42.647745 890 raft_consensus.cc:686] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [term 1 LEADER]: Becoming Leader. State: Replica: a426c8d02e0c489f8f2dc4ec934d321e, State: Running, Role: LEADER
I20260808 15:51:42.647868 890 consensus_queue.cc:237] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } }
I20260808 15:51:42.648500 890 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: TxnStatusTablet state changed. Reason: New leader a426c8d02e0c489f8f2dc4ec934d321e. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.648595 890 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:51:42.648936 940 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 15:51:42.649135 940 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 15:51:42.649361 32650 catalog_manager.cc:5495] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e reported cstate change: term changed from 0 to 1, leader changed from <none> to a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66). New cstate: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } health_report { overall_health: UNKNOWN } } }
I20260808 15:51:42.666303 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.662884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.30.76.66:40821)
0808 15:51:42.663035 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.663040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.663053 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:51:42.663189 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.663191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.663239 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.663341 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:42.663605 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.664090 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.664933 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.665148 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.665254 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.665544 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:42.665922 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:42.666044 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.666119 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.666171 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:51:42.666303 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.662736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40821 (local address 127.0.0.1:36514)
0808 15:51:42.662886 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.662905 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.663033 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.663355 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.663358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.663366 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:42.663473 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.663478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.664203 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.664207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.664809 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.664816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.665255 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.665259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.665352 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.665393 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:42.666020 (+ 627us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:42.666037 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.666109 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.666190 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 15:51:42.668992 789 raft_consensus.cc:1261] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Refusing update from remote peer a426c8d02e0c489f8f2dc4ec934d321e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:42.669162 395 raft_consensus.cc:1261] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Refusing update from remote peer a426c8d02e0c489f8f2dc4ec934d321e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:42.669474 926 consensus_queue.cc:1035] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:42.669700 890 consensus_queue.cc:1035] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:42.671370 886 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: TxnStatusTablet state changed. Reason: New leader a426c8d02e0c489f8f2dc4ec934d321e. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.671393 912 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: TxnStatusTablet state changed. Reason: New leader a426c8d02e0c489f8f2dc4ec934d321e. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.671483 912 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:42.671481 886 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 15:51:42.673065 847 tablet.cc:2369] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:51:42.673617 890 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.673974 890 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:51:42.674681 886 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.674783 886 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P e8108b3987c14bcbb2875833102e8f35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:42.675944 926 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.676040 926 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P a426c8d02e0c489f8f2dc4ec934d321e: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:51:42.678215 912 tablet_replica.cc:439] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 } } peers { permanent_uuid: "a426c8d02e0c489f8f2dc4ec934d321e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40821 } } peers { permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 } } }
I20260808 15:51:42.678326 912 tablet_replica.cc:442] T 90d666687d0f441995bf8bf2f45e8a5a P 06dc8d6d03304f3091f6e339cb58e647: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:42.683204 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.680102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.30.76.66:40821)
0808 15:51:42.680206 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.680211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.680257 (+ 46us) server_negotiation.cc:407] Connection header received
0808 15:51:42.680417 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.680421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.680464 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.680546 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:42.680800 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.681301 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.682115 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.682366 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.682504 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.682756 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:42.682832 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:42.682912 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.683001 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.683048 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:51:42.693668 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.689952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:45269 (local address 127.30.76.66:40821)
0808 15:51:42.690070 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.690074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.690291 (+ 217us) server_negotiation.cc:407] Connection header received
0808 15:51:42.690418 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.690420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.690459 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.690512 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:42.690789 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.691231 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.692551 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.692845 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.692921 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.693244 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:42.693302 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:42.693351 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.693505 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.693556 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:51:42.693701 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.689891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40821 (local address 127.30.76.66:45269)
0808 15:51:42.690203 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:42.690219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:42.690322 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:42.690548 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:42.690551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:42.690558 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:42.690691 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.690697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.691330 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.691334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.692421 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:42.692432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.692947 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.692951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:42.693047 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.693096 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:42.693377 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:42.693383 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:51:42.693449 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 15:51:42.693520 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
I20260808 15:51:42.699793 789 raft_consensus.cc:1261] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Refusing update from remote peer a426c8d02e0c489f8f2dc4ec934d321e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:42.699842 395 raft_consensus.cc:1261] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Refusing update from remote peer a426c8d02e0c489f8f2dc4ec934d321e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:42.700213 951 consensus_queue.cc:1035] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06dc8d6d03304f3091f6e339cb58e647" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:42.700382 926 consensus_queue.cc:1035] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8108b3987c14bcbb2875833102e8f35" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:42.740361 32650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:42.740541 32650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:42.742777 32650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28774ba9f63d4cd7b00012da0bb64c37: Sending DeleteTablet for 3 replicas of tablet 674693dcf0bd443da231743a081d0d97
I20260808 15:51:42.743361 369 tablet_service.cc:1514] Processing DeleteTablet for tablet 674693dcf0bd443da231743a081d0d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:42 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:50250
I20260808 15:51:42.743553 766 tablet_service.cc:1514] Processing DeleteTablet for tablet 674693dcf0bd443da231743a081d0d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:42 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56778
I20260808 15:51:42.743709 916 tablet_replica.cc:330] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: stopping tablet replica
I20260808 15:51:42.743722 917 tablet_replica.cc:330] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: stopping tablet replica
I20260808 15:51:42.743819 917 raft_consensus.cc:2229] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:42.743818 916 raft_consensus.cc:2229] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:42.743918 916 raft_consensus.cc:2258] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:42.743918 917 raft_consensus.cc:2258] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:42.744391 570 tablet_service.cc:1514] Processing DeleteTablet for tablet 674693dcf0bd443da231743a081d0d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:42 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:36510
I20260808 15:51:42.744565 915 tablet_replica.cc:330] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: stopping tablet replica
I20260808 15:51:42.744669 915 raft_consensus.cc:2229] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:51:42.744923 915 raft_consensus.cc:2258] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:42.745589 915 ts_tablet_manager.cc:1905] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:42.747673 917 ts_tablet_manager.cc:1905] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:42.747816 916 ts_tablet_manager.cc:1905] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:42.748546 915 ts_tablet_manager.cc:1918] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:42.748615 915 log.cc:1196] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/674693dcf0bd443da231743a081d0d97
I20260808 15:51:42.748894 915 ts_tablet_manager.cc:1939] T 674693dcf0bd443da231743a081d0d97 P a426c8d02e0c489f8f2dc4ec934d321e: Deleting consensus metadata
I20260808 15:51:42.750475 917 ts_tablet_manager.cc:1918] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:42.750562 917 log.cc:1196] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/674693dcf0bd443da231743a081d0d97
I20260808 15:51:42.750623 916 ts_tablet_manager.cc:1918] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:42.750694 916 log.cc:1196] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/674693dcf0bd443da231743a081d0d97
I20260808 15:51:42.750834 917 ts_tablet_manager.cc:1939] T 674693dcf0bd443da231743a081d0d97 P 06dc8d6d03304f3091f6e339cb58e647: Deleting consensus metadata
I20260808 15:51:42.750941 916 ts_tablet_manager.cc:1939] T 674693dcf0bd443da231743a081d0d97 P e8108b3987c14bcbb2875833102e8f35: Deleting consensus metadata
I20260808 15:51:42.751641 32626 catalog_manager.cc:4841] TS a426c8d02e0c489f8f2dc4ec934d321e (127.30.76.66:40821): tablet 674693dcf0bd443da231743a081d0d97 (table test-table [id=520f3c4022b44d218f71bbfc8f951994]) successfully deleted
I20260808 15:51:42.751753 32629 catalog_manager.cc:4841] TS e8108b3987c14bcbb2875833102e8f35 (127.30.76.67:44679): tablet 674693dcf0bd443da231743a081d0d97 (table test-table [id=520f3c4022b44d218f71bbfc8f951994]) successfully deleted
I20260808 15:51:42.751812 32626 catalog_manager.cc:4841] TS 06dc8d6d03304f3091f6e339cb58e647 (127.30.76.65:35837): tablet 674693dcf0bd443da231743a081d0d97 (table test-table [id=520f3c4022b44d218f71bbfc8f951994]) successfully deleted
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204302, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:42.768069 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.744701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.30.76.65:35837)
0808 15:51:42.744803 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.744807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.755923 (+ 11116us) server_negotiation.cc:407] Connection header received
0808 15:51:42.756070 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.756073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.756121 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.756212 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:42.757083 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.757918 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.759233 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.759596 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.759683 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.763749 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:42.763761 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:42.763778 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:42.763809 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:42.765910 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.766477 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.766484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.766486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.766589 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.766857 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.766860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.766862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.767211 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:42.767511 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.767735 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.767871 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 15:51:42.768596 451 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50264
I20260808 15:51:42.778780 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:42.770595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.30.76.126:37415)
0808 15:51:42.770725 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:42.770731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:42.771107 (+ 376us) server_negotiation.cc:407] Connection header received
0808 15:51:42.771251 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:42.771254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:42.771313 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:42.771404 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:42.772207 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.772749 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.773615 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:42.773844 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:42.773956 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:42.775365 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:42.775369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:42.775372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:42.775407 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:42.776991 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.777477 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.777481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.777483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.777575 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:42.777882 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:42.777885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:42.777887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:42.778080 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:42.778219 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:42.778479 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 15:51:42.778615 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 15:51:42.779433 32650 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36690
I20260808 15:51:42.781332 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 32750
I20260808 15:51:42.796658 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 478
I20260808 15:51:42.808187 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 672
I20260808 15:51:42.817767 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 32604
2026-08-08T15:51:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2869 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[964](info): set up 2 sockets
Aug 08 15:51:42 dist-test-slave-sb2z krb5kdc[964](info): commencing operation
krb5kdc: starting...
W20260808 15:51:44.039669 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:44.162223 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41073
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:44.273502 980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:44.273797 980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:44.273885 980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:44.277851 980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:44.277926 980 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 15:51:44.277956 980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:44.278019 980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:44.278059 980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:44.278097 980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:44.283305 980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41073
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:44.284402 980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:44.285409 980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:44.293790 991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:44.293864 986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:44.293852 989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:44.293922 980 server_base.cc:1034] running on GCE node
I20260808 15:51:44.294718 980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:44.295845 980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:44.297047 980 hybrid_clock.cc:648] HybridClock initialized: now 1786204304297027 us; error 29 us; skew 500 ppm
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:44.299829 980 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:44.300628 980 webserver.cc:460] Webserver started at http://127.30.76.126:39851/ using document root <none> and password file <none>
I20260808 15:51:44.301142 980 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:44.301200 980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:44.301360 980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:44.303308 980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "b2a92f78b35f4edda783bf152e6cf147"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "461e968c027aea784f894cf7ee1b44bc"
server_key_iv: "391c3052d067bbe8f916d9c3b9cb6e21"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.303959 980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "b2a92f78b35f4edda783bf152e6cf147"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "461e968c027aea784f894cf7ee1b44bc"
server_key_iv: "391c3052d067bbe8f916d9c3b9cb6e21"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.307416 980 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:51:44.309707 999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:44.310868 980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:44.311007 980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "b2a92f78b35f4edda783bf152e6cf147"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "461e968c027aea784f894cf7ee1b44bc"
server_key_iv: "391c3052d067bbe8f916d9c3b9cb6e21"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.311121 980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:44.340713 980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:44.344230 980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:44.344440 980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:44.352603 980 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41073
I20260808 15:51:44.352618 1080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41073 every 8 connection(s)
I20260808 15:51:44.353507 980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:44.356325 1081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:44.358678 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 980
I20260808 15:51:44.358774 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:44.359059 31025 external_mini_cluster.cc:1428] Setting key 6c34bca62850c05265a366ddc4316e96
I20260808 15:51:44.362113 1081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Bootstrap starting.
I20260808 15:51:44.364686 1081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:44.365505 1081 log.cc:824] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:44.367367 1081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: No bootstrap required, opened a new log
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:44.371874 1081 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a92f78b35f4edda783bf152e6cf147" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41073 } }
I20260808 15:51:44.372187 1081 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:44.372283 1081 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a92f78b35f4edda783bf152e6cf147, State: Initialized, Role: FOLLOWER
I20260808 15:51:44.372753 1081 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a92f78b35f4edda783bf152e6cf147" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41073 } }
I20260808 15:51:44.372912 1081 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:44.372985 1081 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:44.373080 1081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:44.373490 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.360415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.30.76.126:41073)
0808 15:51:44.360815 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:44.360831 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:44.360865 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:51:44.361494 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:44.361510 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:44.361811 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:44.362075 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:44.362975 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.364020 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.365180 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.365684 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.365809 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.368378 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:44.368385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:44.368400 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:44.368430 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:44.370349 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.370770 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.370777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.370783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.370923 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.371218 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.371223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.371226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.371758 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:44.371967 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:44.372211 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 15:51:44.372516 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":138,"threads_started":1}
I20260808 15:51:44.374118 1081 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a92f78b35f4edda783bf152e6cf147" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41073 } }
I20260808 15:51:44.374490 1081 leader_election.cc:302] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2a92f78b35f4edda783bf152e6cf147; no voters:
I20260808 15:51:44.374850 1081 leader_election.cc:288] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:44.374960 1087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:44.375211 1087 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [term 1 LEADER]: Becoming Leader. State: Replica: b2a92f78b35f4edda783bf152e6cf147, State: Running, Role: LEADER
I20260808 15:51:44.375631 1087 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a92f78b35f4edda783bf152e6cf147" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41073 } }
I20260808 15:51:44.376128 1081 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:44.377247 1091 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2a92f78b35f4edda783bf152e6cf147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a92f78b35f4edda783bf152e6cf147" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41073 } } }
I20260808 15:51:44.377375 1091 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:44.377848 1100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:44.377815 1089 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a92f78b35f4edda783bf152e6cf147. Latest consensus state: current_term: 1 leader_uuid: "b2a92f78b35f4edda783bf152e6cf147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a92f78b35f4edda783bf152e6cf147" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41073 } } }
I20260808 15:51:44.378618 1089 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:44.380728 1100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:44.386179 1100 catalog_manager.cc:1347] Generated new cluster ID: 0d7ff58291254937bcaef338dec0df8b
I20260808 15:51:44.386250 1100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:44.392987 1100 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:44.394171 1100 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:44.406821 1100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Generated new TSK 0
I20260808 15:51:44.407634 1100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:44.446014 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41073
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:44.573254 1119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:44.573491 1119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:44.573572 1119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:44.578316 1119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:44.578374 1119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:44.578439 1119 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 15:51:44.578470 1119 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:51:44.578523 1119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:44.583705 1119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:41073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:44.584674 1119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:44.585673 1119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:44.595079 1125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:44.595093 1130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:44.595081 1126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:44.595357 1119 server_base.cc:1034] running on GCE node
I20260808 15:51:44.595858 1119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:44.596577 1119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:44.597738 1119 hybrid_clock.cc:648] HybridClock initialized: now 1786204304597723 us; error 25 us; skew 500 ppm
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:44.600461 1119 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:44.601292 1119 webserver.cc:460] Webserver started at http://127.30.76.65:35743/ using document root <none> and password file <none>
I20260808 15:51:44.601820 1119 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:44.601879 1119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:44.602133 1119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:44.604229 1119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "99a1939cc70945178b0f4455b5b0d574"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "4b4b947d07a6fb1d373e41e10e273a80"
server_key_iv: "b92f7b5f49c0740fc2b152a47116c504"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.604789 1119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "99a1939cc70945178b0f4455b5b0d574"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "4b4b947d07a6fb1d373e41e10e273a80"
server_key_iv: "b92f7b5f49c0740fc2b152a47116c504"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.608382 1119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:44.610591 1142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:44.611801 1119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:44.611936 1119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "99a1939cc70945178b0f4455b5b0d574"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "4b4b947d07a6fb1d373e41e10e273a80"
server_key_iv: "b92f7b5f49c0740fc2b152a47116c504"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.612043 1119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:44.630614 1119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:44.633546 1119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:44.633752 1119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:44.635172 1119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:44.635246 1119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:44.635322 1119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:44.635365 1119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:44.646341 1119 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:36307
I20260808 15:51:44.646597 1297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:36307 every 8 connection(s)
I20260808 15:51:44.647446 1119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:44.649953 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.636067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:49513 (local address 127.30.76.126:41073)
0808 15:51:44.636201 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:44.636205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:44.637247 (+ 1042us) server_negotiation.cc:407] Connection header received
0808 15:51:44.638210 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:44.638217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:44.638290 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:44.638387 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:44.640104 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.640971 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.642220 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.642625 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.642752 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.646570 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:44.646575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:44.646577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:44.646612 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:44.648176 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.648704 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.648707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.648708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.648786 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.649050 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.649052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.649053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.649226 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:44.649303 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:44.649732 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 15:51:44.649834 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:51:44.650728 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.636600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.30.76.65:49513)
0808 15:51:44.637099 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:44.637139 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:44.637951 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:44.638662 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:44.638674 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:44.639219 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:44.639933 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.639956 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.641088 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.641094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.642034 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.642069 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.642829 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.642836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.643076 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.644440 (+ 1364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:44.644466 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:44.646360 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:44.648281 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.648288 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.648300 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.648609 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.648849 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.648850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.648852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.648970 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.649323 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:44.649327 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:44.649534 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 15:51:44.649681 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:51:44.649906 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":148,"threads_started":1}
I20260808 15:51:44.652153 1299 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41073
I20260808 15:51:44.652307 1299 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:44.652789 1299 heartbeater.cc:511] Master 127.30.76.126:41073 requested a full tablet report, sending...
I20260808 15:51:44.653539 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 1119
I20260808 15:51:44.653666 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:44.653966 31025 external_mini_cluster.cc:1428] Setting key 6161be572d8cd1371d146bcb240d10aa
I20260808 15:51:44.655165 1018 ts_manager.cc:194] Registered new tserver with Master: 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307)
I20260808 15:51:44.656780 1018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:49513
I20260808 15:51:44.661339 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.654430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:45365 (local address 127.30.76.126:41073)
0808 15:51:44.654541 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:44.654545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:44.654625 (+ 80us) server_negotiation.cc:407] Connection header received
0808 15:51:44.654808 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:44.654811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:44.654857 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:44.654937 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:44.655468 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.656020 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.656829 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.657097 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.657208 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.658226 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:44.658229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:44.658232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:44.658255 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:44.659735 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.660160 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.660165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.660167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.660242 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.660524 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.660528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.660530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.660695 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:44.660803 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:44.661084 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 15:51:44.661198 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:51:44.661348 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.654283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.30.76.65:45365)
0808 15:51:44.654533 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:44.654545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:44.654702 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:44.654974 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:44.654977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:44.655188 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:44.655352 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.655359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.656151 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.656154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.656703 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.656711 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.657213 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.657216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.657306 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.657777 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:44.657788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:44.658091 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:44.659845 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.659851 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.659853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.660052 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.660337 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.660340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.660342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.660432 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.660794 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:44.660797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:44.660895 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 15:51:44.661015 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 15:51:44.661149 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:44.681950 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41073
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:44.798197 1312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:44.798432 1312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:44.798527 1312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:44.802603 1312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:44.802661 1312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:44.802740 1312 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 15:51:44.802774 1312 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:51:44.802806 1312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:44.808063 1312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:41073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:44.809053 1312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:44.810093 1312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:44.819932 1321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:44.819927 1323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:44.819921 1320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:44.820220 1312 server_base.cc:1034] running on GCE node
I20260808 15:51:44.820730 1312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:44.821522 1312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:44.822695 1312 hybrid_clock.cc:648] HybridClock initialized: now 1786204304822679 us; error 29 us; skew 500 ppm
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:44.825534 1312 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:44.826400 1312 webserver.cc:460] Webserver started at http://127.30.76.66:35015/ using document root <none> and password file <none>
I20260808 15:51:44.826977 1312 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:44.827068 1312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:44.827322 1312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:44.829447 1312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "05687940ea4343158a5c3d43e3109519"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "b56eabbb1c9b4e961a2980b9f7ffded3"
server_key_iv: "a1df4f67a895b65d6f5bf9548628c291"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.830037 1312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "05687940ea4343158a5c3d43e3109519"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "b56eabbb1c9b4e961a2980b9f7ffded3"
server_key_iv: "a1df4f67a895b65d6f5bf9548628c291"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.833796 1312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:51:44.836236 1334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:44.837402 1312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:44.837550 1312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "05687940ea4343158a5c3d43e3109519"
format_stamp: "Formatted at 2026-08-08 15:51:44 on dist-test-slave-sb2z"
server_key: "b56eabbb1c9b4e961a2980b9f7ffded3"
server_key_iv: "a1df4f67a895b65d6f5bf9548628c291"
server_key_version: "encryptionkey@0"
I20260808 15:51:44.837667 1312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:44.871978 1312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:44.875052 1312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:44.875267 1312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:44.876969 1312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:44.877033 1312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:44.877100 1312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:44.877142 1312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:51:44 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:44.887697 1312 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:44625
I20260808 15:51:44.888212 1486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:44625 every 8 connection(s)
I20260808 15:51:44.888681 1312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:44.888854 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 1312
I20260808 15:51:44.888974 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:44.889310 31025 external_mini_cluster.cc:1428] Setting key 9f44819136b164bc3003aa93ddd5f4f9
I20260808 15:51:44.891795 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.877845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:41921 (local address 127.30.76.126:41073)
0808 15:51:44.877966 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:44.877973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:44.879133 (+ 1160us) server_negotiation.cc:407] Connection header received
0808 15:51:44.880344 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:44.880350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:44.880418 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:44.880523 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:44.882037 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.882920 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.884654 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.885020 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.885144 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.887896 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:44.887899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:44.887901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:44.887926 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:44.889523 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.890106 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.890109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.890111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.890208 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.890596 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.890600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.890602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.890863 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:44.890983 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:44.891462 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 15:51:44.891621 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:51:44.892658 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.878412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.30.76.66:41921)
0808 15:51:44.878989 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:44.879031 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:44.880071 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:44.880720 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:44.880731 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:44.881160 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:44.881853 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.881874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.883058 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.883061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.884513 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.884526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.885184 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.885187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.885320 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.885901 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:44.885916 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:44.887700 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:44.889654 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.889659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.889671 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.890007 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.890308 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.890311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.890313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.890510 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.890981 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:44.890988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:44.891225 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 15:51:44.891403 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 15:51:44.891730 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":148,"threads_started":1}
I20260808 15:51:44.894068 1488 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41073
I20260808 15:51:44.894214 1488 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:44.894764 1488 heartbeater.cc:511] Master 127.30.76.126:41073 requested a full tablet report, sending...
I20260808 15:51:44.895856 1018 ts_manager.cc:194] Registered new tserver with Master: 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625)
I20260808 15:51:44.896580 1018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:41921
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:51:44.904605 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.896997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:45963 (local address 127.30.76.126:41073)
0808 15:51:44.897373 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:44.897377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:44.897389 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:51:44.897563 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:44.897566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:44.897610 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:44.897662 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:44.898306 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.899172 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.900052 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.900269 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.900383 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.901513 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:44.901516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:44.901519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:44.901549 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:44.902800 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.903269 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.903272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.903273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.903383 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:44.903705 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:44.903707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:44.903708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:44.904042 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:44.904167 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:44.904402 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 15:51:44.904495 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296}
I20260808 15:51:44.904736 1499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:44.896304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.30.76.66:45963)
0808 15:51:44.897288 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:44.897306 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:44.897393 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:44.897799 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:44.897802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:44.898005 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:44.898163 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.898176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.899305 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.899308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.899897 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:44.899906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:44.900389 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:44.900393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:44.900484 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:44.900994 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:44.901005 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:44.901381 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:44.902913 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.902918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.902920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.903144 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.903471 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:44.903473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:44.903474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:44.903622 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:44.904154 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:44.904156 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:44.904231 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 15:51:44.904315 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 15:51:44.904479 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":1,"thread_start_us":747,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:44.919826 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41073
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:45.034055 1503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:45.034354 1503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:45.034438 1503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:45.038398 1503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:45.038493 1503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:45.038571 1503 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 15:51:45.038617 1503 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:51:45.038686 1503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:45.043954 1503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:41073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:45.044991 1503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:45.046015 1503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:45.055568 1515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:45.055733 1503 server_base.cc:1034] running on GCE node
W20260808 15:51:45.055498 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:45.055590 1513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:45.056241 1503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:45.056869 1503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:45.058054 1503 hybrid_clock.cc:648] HybridClock initialized: now 1786204305058019 us; error 47 us; skew 500 ppm
Aug 08 15:51:45 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:45.061120 1503 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:45.061964 1503 webserver.cc:460] Webserver started at http://127.30.76.67:35765/ using document root <none> and password file <none>
I20260808 15:51:45.062547 1503 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:45.062639 1503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:45.062857 1503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:45.065236 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "74e30f339dde483eba4cb174a0c1eb4d"
format_stamp: "Formatted at 2026-08-08 15:51:45 on dist-test-slave-sb2z"
server_key: "94fc379b3e9042b796e3b53a2560f470"
server_key_iv: "d55303e4a4eeab119a4d398293a072fe"
server_key_version: "encryptionkey@0"
I20260808 15:51:45.065856 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "74e30f339dde483eba4cb174a0c1eb4d"
format_stamp: "Formatted at 2026-08-08 15:51:45 on dist-test-slave-sb2z"
server_key: "94fc379b3e9042b796e3b53a2560f470"
server_key_iv: "d55303e4a4eeab119a4d398293a072fe"
server_key_version: "encryptionkey@0"
I20260808 15:51:45.069564 1503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:51:45.071895 1523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:45.073071 1503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:45.073230 1503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "74e30f339dde483eba4cb174a0c1eb4d"
format_stamp: "Formatted at 2026-08-08 15:51:45 on dist-test-slave-sb2z"
server_key: "94fc379b3e9042b796e3b53a2560f470"
server_key_iv: "d55303e4a4eeab119a4d398293a072fe"
server_key_version: "encryptionkey@0"
I20260808 15:51:45.073344 1503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:45.093801 1503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:45.097251 1503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:45.097460 1503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:45.098887 1503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:45.098968 1503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:45.099045 1503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:45.099090 1503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:51:45 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:45.110419 1503 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:38645
I20260808 15:51:45.110559 1679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:38645 every 8 connection(s)
I20260808 15:51:45.111367 1503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:45.113587 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.099776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:42023 (local address 127.30.76.126:41073)
0808 15:51:45.099902 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.099906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.100954 (+ 1048us) server_negotiation.cc:407] Connection header received
0808 15:51:45.101943 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.101949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.102017 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.102121 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:45.103745 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.104664 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.106470 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.106728 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.106846 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.109776 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:45.109779 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:45.109781 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:45.109805 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:45.111470 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.112111 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.112117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.112120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.112225 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.112589 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.112591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.112592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.112766 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:45.112845 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.113297 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.113390 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:51:45.114635 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.100322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.30.76.67:42023)
0808 15:51:45.100802 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.100841 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.101702 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.102394 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.102403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.102892 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:45.103568 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.103588 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.104811 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.104815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.106337 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.106348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.106898 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.106902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.107093 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.107785 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:45.107800 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:45.109594 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:45.111606 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:45.111611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:45.111624 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:45.111987 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:45.112313 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:45.112316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:45.112318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:45.112481 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:45.112861 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:45.112866 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:45.113075 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.113247 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.113528 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260808 15:51:45.115916 1680 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41073
I20260808 15:51:45.116050 1680 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:45.116245 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 1503
I20260808 15:51:45.116374 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:45.116575 1680 heartbeater.cc:511] Master 127.30.76.126:41073 requested a full tablet report, sending...
I20260808 15:51:45.116701 31025 external_mini_cluster.cc:1428] Setting key bed61db114ba689dbcc99f100f4ade5a
I20260808 15:51:45.117734 1018 ts_manager.cc:194] Registered new tserver with Master: 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:45.118495 1018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:42023
I20260808 15:51:45.118971 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:51:45.126106 1691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.118332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.30.76.67:51585)
0808 15:51:45.118604 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.118619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.118725 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.119005 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.119010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.119234 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:45.119349 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.119356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.120367 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.120374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.121340 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.121356 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.121985 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.121988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.122054 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.122542 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:45.122561 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:45.122857 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:45.124559 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:45.124562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:45.124565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:45.124779 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:45.125073 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:45.125076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:45.125078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:45.125167 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:45.125575 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:45.125580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:45.125680 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.125790 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.125930 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260808 15:51:45.126094 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.118360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:51585 (local address 127.30.76.126:41073)
0808 15:51:45.118599 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.118603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.118685 (+ 82us) server_negotiation.cc:407] Connection header received
0808 15:51:45.118851 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.118854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.118899 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.118983 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:45.119478 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.120221 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.121487 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.121860 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.121976 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.122987 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:45.122989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:45.122991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:45.123013 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:45.124433 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.124889 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.124893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.124894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.124976 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.125274 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.125278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.125280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.125462 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:45.125568 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.125848 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.125957 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
Aug 08 15:51:45 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:51:45 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:45.146999 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.138864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.30.76.126:41073)
0808 15:51:45.139015 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.139019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.139031 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:51:45.139066 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.139068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.139105 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.139203 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:45.139960 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.140464 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.141395 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.141638 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.141766 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.144097 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:45.144101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:45.144104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:45.144140 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:45.145311 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.145749 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.145752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.145754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.145837 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.146124 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.146129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.146134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.146316 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:45.146461 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.146712 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.146828 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:51:45.149916 1018 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:51:45.153054 1018 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:45.167057 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.161315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:36307 (local address 127.0.0.1:45146)
0808 15:51:45.161686 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.161738 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.161868 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.162854 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.162860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.162888 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.163049 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.163060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.164535 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.164538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.165595 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.165605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.166640 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.166643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.166727 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.166752 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.166798 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.166939 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":19,"thread_start_us":104,"threads_started":1}
I20260808 15:51:45.167068 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.162239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.30.76.67:38645)
0808 15:51:45.162676 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.162688 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.162714 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:51:45.162793 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.162800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.162986 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.163168 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.163580 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.164602 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.165862 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.166369 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.166485 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.166642 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.166825 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.166921 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":116,"threads_started":1}
I20260808 15:51:45.167340 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.161459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.30.76.65:36307)
0808 15:51:45.162327 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.162335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.162360 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:51:45.162445 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.162450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.162681 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.163209 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.163333 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.164408 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.165729 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.166503 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.166925 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.167064 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.167126 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.167211 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":77,"threads_started":1}
I20260808 15:51:45.167749 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.161605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38645 (local address 127.0.0.1:53222)
0808 15:51:45.161845 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.161858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.161935 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.163317 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.163320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.163337 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.163421 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.163427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.164710 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.164714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.165740 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.165750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.166498 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.166501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.166601 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.166652 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.166792 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.167620 (+ 828us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260808 15:51:45.168398 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.162104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:44625 (local address 127.0.0.1:59210)
0808 15:51:45.162759 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.162772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.162854 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.163738 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.163740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.163755 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.163832 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.163836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.165086 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.165089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.166771 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.166782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.167975 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.167982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.168121 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.168144 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.168214 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.168281 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":106,"threads_started":1}
I20260808 15:51:45.168752 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.162284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.30.76.66:44625)
0808 15:51:45.163145 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.163156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.163177 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:51:45.163260 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.163264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.163428 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.163602 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.163951 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.164959 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.166914 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.167701 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.167817 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.167998 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.168486 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.168586 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":114,"threads_started":1}
I20260808 15:51:45.170429 1591 tablet_service.cc:1467] Processing CreateTablet for tablet d808ceeb32924917a739c3b996082fe7 (DEFAULT_TABLE table=test-table [id=e3b960f0c79f4fa08e3dfb528cc205eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:45.170359 1209 tablet_service.cc:1467] Processing CreateTablet for tablet d808ceeb32924917a739c3b996082fe7 (DEFAULT_TABLE table=test-table [id=e3b960f0c79f4fa08e3dfb528cc205eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:45.171283 1591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d808ceeb32924917a739c3b996082fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.171391 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d808ceeb32924917a739c3b996082fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.171200 1396 tablet_service.cc:1467] Processing CreateTablet for tablet d808ceeb32924917a739c3b996082fe7 (DEFAULT_TABLE table=test-table [id=e3b960f0c79f4fa08e3dfb528cc205eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:45.172122 1396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d808ceeb32924917a739c3b996082fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.177907 1713 tablet_bootstrap.cc:492] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Bootstrap starting.
I20260808 15:51:45.179421 1715 tablet_bootstrap.cc:492] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Bootstrap starting.
I20260808 15:51:45.179988 1714 tablet_bootstrap.cc:492] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Bootstrap starting.
I20260808 15:51:45.180459 1713 tablet_bootstrap.cc:654] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.181330 1713 log.cc:824] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:45.181615 1715 tablet_bootstrap.cc:654] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.181931 1714 tablet_bootstrap.cc:654] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.182375 1715 log.cc:824] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:45.182667 1714 log.cc:824] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:45.184291 1715 tablet_bootstrap.cc:492] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: No bootstrap required, opened a new log
I20260808 15:51:45.184516 1715 ts_tablet_manager.cc:1397] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:51:45.184641 1714 tablet_bootstrap.cc:492] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: No bootstrap required, opened a new log
I20260808 15:51:45.184641 1713 tablet_bootstrap.cc:492] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: No bootstrap required, opened a new log
I20260808 15:51:45.184829 1714 ts_tablet_manager.cc:1397] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 15:51:45.184829 1713 ts_tablet_manager.cc:1397] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 15:51:45.187798 1715 raft_consensus.cc:348] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.188033 1715 raft_consensus.cc:374] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.188088 1715 raft_consensus.cc:729] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.188548 1715 consensus_queue.cc:260] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.188763 1713 raft_consensus.cc:348] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.189011 1713 raft_consensus.cc:374] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.189079 1713 raft_consensus.cc:729] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e30f339dde483eba4cb174a0c1eb4d, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.189188 1714 raft_consensus.cc:348] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.189368 1488 heartbeater.cc:503] Master 127.30.76.126:41073 was elected leader, sending a full tablet report...
I20260808 15:51:45.189415 1714 raft_consensus.cc:374] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.189463 1714 raft_consensus.cc:729] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a1939cc70945178b0f4455b5b0d574, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.189515 1713 consensus_queue.cc:260] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.189602 1715 ts_tablet_manager.cc:1428] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 15:51:45.189883 1714 consensus_queue.cc:260] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.190501 1299 heartbeater.cc:503] Master 127.30.76.126:41073 was elected leader, sending a full tablet report...
I20260808 15:51:45.190750 1713 ts_tablet_manager.cc:1428] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:51:45.190750 1714 ts_tablet_manager.cc:1428] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:51:45.191069 1680 heartbeater.cc:503] Master 127.30.76.126:41073 was elected leader, sending a full tablet report...
I20260808 15:51:45.345467 1728 raft_consensus.cc:482] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:45.345707 1728 raft_consensus.cc:504] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.346977 1728 leader_election.cc:288] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:45.351121 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.347424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39057 (local address 127.30.76.67:38645)
0808 15:51:45.347576 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.347581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.347714 (+ 133us) server_negotiation.cc:407] Connection header received
0808 15:51:45.347895 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.347898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.347948 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.348033 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.348311 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.348991 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.350146 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.350550 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.350681 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.350735 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.350909 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.350987 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:51:45.351136 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.347212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38645 (local address 127.30.76.65:39057)
0808 15:51:45.347601 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.347615 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.347771 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.348057 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.348060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.348079 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.348203 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.348209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.349104 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.349110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.350016 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.350026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.350676 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.350679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.350774 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.350797 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.350893 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.350962 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":125,"threads_started":1}
I20260808 15:51:45.351164 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.347331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39653 (local address 127.30.76.66:44625)
0808 15:51:45.347462 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.347468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.347968 (+ 500us) server_negotiation.cc:407] Connection header received
0808 15:51:45.348126 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.348129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.348192 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.348274 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.348537 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.349208 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.350208 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.350597 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.350724 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.350776 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.350957 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.351025 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:51:45.351191 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.347212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:44625 (local address 127.30.76.65:39653)
0808 15:51:45.347868 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.347880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.348009 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.348284 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.348287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.348303 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.348430 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.348435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.349339 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.349342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.350082 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.350091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.350717 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.350720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.350826 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.350849 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.350943 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.351012 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":257,"thread_start_us":139,"threads_started":1}
I20260808 15:51:45.352018 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d808ceeb32924917a739c3b996082fe7" candidate_uuid: "99a1939cc70945178b0f4455b5b0d574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d" is_pre_election: true
I20260808 15:51:45.352145 1426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d808ceeb32924917a739c3b996082fe7" candidate_uuid: "99a1939cc70945178b0f4455b5b0d574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05687940ea4343158a5c3d43e3109519" is_pre_election: true
I20260808 15:51:45.352381 1613 raft_consensus.cc:2454] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a1939cc70945178b0f4455b5b0d574 in term 0.
I20260808 15:51:45.352381 1426 raft_consensus.cc:2454] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a1939cc70945178b0f4455b5b0d574 in term 0.
I20260808 15:51:45.352882 1147 leader_election.cc:302] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:45.353173 1728 raft_consensus.cc:2781] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:45.353257 1728 raft_consensus.cc:482] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:45.353296 1728 raft_consensus.cc:3037] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.354588 1728 raft_consensus.cc:504] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.354924 1728 leader_election.cc:288] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [CANDIDATE]: Term 1 election: Requested vote from peers 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:45.355279 1426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d808ceeb32924917a739c3b996082fe7" candidate_uuid: "99a1939cc70945178b0f4455b5b0d574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05687940ea4343158a5c3d43e3109519"
I20260808 15:51:45.355283 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d808ceeb32924917a739c3b996082fe7" candidate_uuid: "99a1939cc70945178b0f4455b5b0d574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d"
I20260808 15:51:45.355373 1426 raft_consensus.cc:3037] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.355396 1613 raft_consensus.cc:3037] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.356652 1426 raft_consensus.cc:2454] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a1939cc70945178b0f4455b5b0d574 in term 1.
I20260808 15:51:45.356663 1613 raft_consensus.cc:2454] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a1939cc70945178b0f4455b5b0d574 in term 1.
I20260808 15:51:45.356932 1147 leader_election.cc:302] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:45.357110 1728 raft_consensus.cc:2781] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:45.357347 1728 raft_consensus.cc:686] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 1 LEADER]: Becoming Leader. State: Replica: 99a1939cc70945178b0f4455b5b0d574, State: Running, Role: LEADER
I20260808 15:51:45.357664 1728 consensus_queue.cc:237] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.360952 1020 catalog_manager.cc:5495] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65). New cstate: current_term: 1 leader_uuid: "99a1939cc70945178b0f4455b5b0d574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } health_report { overall_health: UNKNOWN } } }
W20260808 15:51:45.363402 1682 tablet.cc:2369] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:51:45.390025 1490 tablet.cc:2369] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:51:45.398947 1300 tablet.cc:2369] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 15:51:45 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:45.433507 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.425266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.30.76.126:41073)
0808 15:51:45.425424 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.425429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.425609 (+ 180us) server_negotiation.cc:407] Connection header received
0808 15:51:45.425775 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.425780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.425821 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.425901 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:45.426585 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.427164 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.428009 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.428224 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.428333 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.430496 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:45.430500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:45.430504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:45.430530 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:45.431804 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.432265 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.432270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.432272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.432349 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:45.432634 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:45.432637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:45.432639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:45.432806 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:45.432916 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.433205 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.433364 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":50}
I20260808 15:51:45.433507 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.425228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41073 (local address 127.0.0.1:35380)
0808 15:51:45.425521 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.425533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.425672 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.425921 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.425924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.426346 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:45.426463 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.426470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.427272 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.427277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.427891 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.427900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.428336 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.428339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.428420 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.428948 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:45.428961 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:45.430344 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:45.431909 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:45.431920 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:45.431927 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:45.432163 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:45.432449 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:45.432452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:45.432454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:45.432535 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:45.432916 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:45.432924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:45.433037 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.433156 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.433308 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":102,"threads_started":1}
I20260808 15:51:45.437492 1020 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:35380:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 15:51:45.438189 1020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:45.443188 1396 tablet_service.cc:1467] Processing CreateTablet for tablet 95b001f5b4524325bac22e8d76596af5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3ace53deb37d48e78ef8fe0e5c4c774c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:51:45.443207 1209 tablet_service.cc:1467] Processing CreateTablet for tablet 95b001f5b4524325bac22e8d76596af5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3ace53deb37d48e78ef8fe0e5c4c774c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:51:45.443233 1591 tablet_service.cc:1467] Processing CreateTablet for tablet 95b001f5b4524325bac22e8d76596af5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3ace53deb37d48e78ef8fe0e5c4c774c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:51:45.443955 1396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b001f5b4524325bac22e8d76596af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.443951 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b001f5b4524325bac22e8d76596af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.443951 1591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b001f5b4524325bac22e8d76596af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.446305 1715 tablet_bootstrap.cc:492] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: Bootstrap starting.
I20260808 15:51:45.446826 1714 tablet_bootstrap.cc:492] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: Bootstrap starting.
I20260808 15:51:45.447187 1713 tablet_bootstrap.cc:492] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: Bootstrap starting.
I20260808 15:51:45.447340 1715 tablet_bootstrap.cc:654] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.448163 1713 tablet_bootstrap.cc:654] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.448287 1714 tablet_bootstrap.cc:654] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.448740 1715 tablet_bootstrap.cc:492] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: No bootstrap required, opened a new log
I20260808 15:51:45.448838 1715 ts_tablet_manager.cc:1397] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:51:45.449378 1713 tablet_bootstrap.cc:492] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: No bootstrap required, opened a new log
I20260808 15:51:45.449483 1713 ts_tablet_manager.cc:1397] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:45.449535 1715 raft_consensus.cc:348] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.449663 1715 raft_consensus.cc:374] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.449698 1715 raft_consensus.cc:729] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.449831 1715 consensus_queue.cc:260] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.449908 1713 raft_consensus.cc:348] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.450022 1713 raft_consensus.cc:374] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.450055 1713 raft_consensus.cc:729] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e30f339dde483eba4cb174a0c1eb4d, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.450093 1715 ts_tablet_manager.cc:1428] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 15:51:45.450193 1713 consensus_queue.cc:260] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.450459 1725 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.450595 1725 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.450587 1723 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.450708 1723 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.450472 1713 ts_tablet_manager.cc:1428] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:45.451010 1714 tablet_bootstrap.cc:492] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: No bootstrap required, opened a new log
I20260808 15:51:45.451122 1714 ts_tablet_manager.cc:1397] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 15:51:45.451601 1714 raft_consensus.cc:348] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.451706 1714 raft_consensus.cc:374] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.451730 1714 raft_consensus.cc:729] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a1939cc70945178b0f4455b5b0d574, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.451851 1714 consensus_queue.cc:260] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.452121 1714 ts_tablet_manager.cc:1428] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:45.452082 1728 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.452231 1728 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.560767 1725 raft_consensus.cc:482] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:45.560906 1725 raft_consensus.cc:504] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.562007 1725 leader_election.cc:288] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625)
I20260808 15:51:45.565968 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.562297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:36307 (local address 127.30.76.67:60145)
0808 15:51:45.562452 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.562488 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.562596 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.563099 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.563101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.563120 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.563275 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.563281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.564058 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.564065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.564832 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.564841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.565532 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.565535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.565641 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.565670 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.565760 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.565830 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"spinlock_wait_cycles":20096}
I20260808 15:51:45.565968 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.562427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:60145 (local address 127.30.76.65:36307)
0808 15:51:45.562576 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.562580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.562597 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:51:45.562929 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.562935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.562995 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.563084 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.563393 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.563938 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.564967 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.565405 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.565538 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.565593 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.565778 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.565853 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:51:45.566402 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.562784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37817 (local address 127.30.76.66:44625)
0808 15:51:45.562921 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.562926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.563129 (+ 203us) server_negotiation.cc:407] Connection header received
0808 15:51:45.563316 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.563319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.563369 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.563452 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.563765 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.564260 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.565282 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.565682 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.565822 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.565870 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.566224 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.566296 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:51:45.566428 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.562625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:44625 (local address 127.30.76.67:37817)
0808 15:51:45.563024 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.563036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.563197 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.563463 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.563466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.563482 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.563639 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.563645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.564378 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.564382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.565144 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.565153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.565827 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.565831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.565932 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.565956 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.566203 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.566275 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"spinlock_wait_cycles":768,"thread_start_us":179,"threads_started":1}
I20260808 15:51:45.566679 1234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b001f5b4524325bac22e8d76596af5" candidate_uuid: "74e30f339dde483eba4cb174a0c1eb4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574" is_pre_election: true
I20260808 15:51:45.566964 1234 raft_consensus.cc:2454] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e30f339dde483eba4cb174a0c1eb4d in term 0.
I20260808 15:51:45.566978 1426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b001f5b4524325bac22e8d76596af5" candidate_uuid: "74e30f339dde483eba4cb174a0c1eb4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05687940ea4343158a5c3d43e3109519" is_pre_election: true
I20260808 15:51:45.567092 1426 raft_consensus.cc:2454] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e30f339dde483eba4cb174a0c1eb4d in term 0.
I20260808 15:51:45.567410 1529 leader_election.cc:302] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e30f339dde483eba4cb174a0c1eb4d, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:45.567750 1725 raft_consensus.cc:2781] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:45.567831 1725 raft_consensus.cc:482] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:45.567880 1725 raft_consensus.cc:3037] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.569119 1725 raft_consensus.cc:504] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.569465 1725 leader_election.cc:288] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [CANDIDATE]: Term 1 election: Requested vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625)
I20260808 15:51:45.569823 1234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b001f5b4524325bac22e8d76596af5" candidate_uuid: "74e30f339dde483eba4cb174a0c1eb4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574"
I20260808 15:51:45.569921 1234 raft_consensus.cc:3037] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.569880 1426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b001f5b4524325bac22e8d76596af5" candidate_uuid: "74e30f339dde483eba4cb174a0c1eb4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05687940ea4343158a5c3d43e3109519"
I20260808 15:51:45.570020 1426 raft_consensus.cc:3037] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.570895 1234 raft_consensus.cc:2454] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e30f339dde483eba4cb174a0c1eb4d in term 1.
I20260808 15:51:45.570897 1426 raft_consensus.cc:2454] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e30f339dde483eba4cb174a0c1eb4d in term 1.
I20260808 15:51:45.571161 1529 leader_election.cc:302] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e30f339dde483eba4cb174a0c1eb4d, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:45.571349 1725 raft_consensus.cc:2781] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:45.571616 1725 raft_consensus.cc:686] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 LEADER]: Becoming Leader. State: Replica: 74e30f339dde483eba4cb174a0c1eb4d, State: Running, Role: LEADER
I20260808 15:51:45.571952 1725 consensus_queue.cc:237] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.574292 1744 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: TxnStatusTablet state changed. Reason: New leader 74e30f339dde483eba4cb174a0c1eb4d. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.574406 1744 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:51:45.574693 1747 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 15:51:45.574923 1747 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 15:51:45.575191 1020 catalog_manager.cc:5495] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67). New cstate: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } health_report { overall_health: HEALTHY } } }
I20260808 15:51:45.608326 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.604440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38645 (local address 127.0.0.1:53236)
0808 15:51:45.604694 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.604712 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.604825 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.605170 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.605172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.605180 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:45.605301 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.605312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.606046 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.606050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.606673 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.606682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.607141 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.607145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.607233 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.607285 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.608001 (+ 716us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.608018 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.608091 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.608186 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 15:51:45.608326 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.604564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.30.76.67:38645)
0808 15:51:45.604716 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.604722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.604757 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:51:45.604990 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.604993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.605047 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.605149 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.605435 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.605931 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.606798 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.607016 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.607124 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.607440 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.607896 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.608026 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.608143 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.608193 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:51:45.613453 1426 raft_consensus.cc:1261] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Refusing update from remote peer 74e30f339dde483eba4cb174a0c1eb4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:45.613451 1234 raft_consensus.cc:1261] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Refusing update from remote peer 74e30f339dde483eba4cb174a0c1eb4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:45.614120 1725 consensus_queue.cc:1035] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:45.614316 1744 consensus_queue.cc:1035] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:45.617762 1728 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: TxnStatusTablet state changed. Reason: New leader 74e30f339dde483eba4cb174a0c1eb4d. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.617901 1728 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.618933 1723 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: TxnStatusTablet state changed. Reason: New leader 74e30f339dde483eba4cb174a0c1eb4d. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.619050 1723 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.621505 1750 mvcc.cc:205] Tried to move back new op lower bound from 7316292835786141696 to 7316292835625455616. Current Snapshot: MvccSnapshot[applied={T|T < 7316292835786141696}]
I20260808 15:51:45.621693 1728 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.621795 1728 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 99a1939cc70945178b0f4455b5b0d574: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.623612 1723 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.623716 1723 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 05687940ea4343158a5c3d43e3109519: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:51:45.624420 1744 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.624540 1744 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:51:45.621522 1758 tablet_replica.cc:439] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74e30f339dde483eba4cb174a0c1eb4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } } }
I20260808 15:51:45.625268 1758 tablet_replica.cc:442] T 95b001f5b4524325bac22e8d76596af5 P 74e30f339dde483eba4cb174a0c1eb4d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:51:45.626631 1753 mvcc.cc:205] Tried to move back new op lower bound from 7316292835786141696 to 7316292835625455616. Current Snapshot: MvccSnapshot[applied={T|T < 7316292835786141696 or (T in {7316292835786141696})}]
I20260808 15:51:45.629124 1754 mvcc.cc:205] Tried to move back new op lower bound from 7316292835786141696 to 7316292835625455616. Current Snapshot: MvccSnapshot[applied={T|T < 7316292835786141696}]
I20260808 15:51:45.634218 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.630433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.30.76.65:36307)
0808 15:51:45.630552 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.630556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.630607 (+ 51us) server_negotiation.cc:407] Connection header received
0808 15:51:45.630743 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.630746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.630805 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.630912 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.631136 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.631937 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.632746 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.633101 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.633205 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.633428 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.633884 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.634001 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.634042 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.634094 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:51:45.644693 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.641324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:59129 (local address 127.30.76.67:38645)
0808 15:51:45.641454 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.641458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.641699 (+ 241us) server_negotiation.cc:407] Connection header received
0808 15:51:45.641844 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.641847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.641902 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.641967 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.642248 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.642725 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.643641 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.643862 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.643976 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.644282 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.644350 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.644430 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.644520 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.644564 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:51:45.644742 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.641254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38645 (local address 127.30.76.65:59129)
0808 15:51:45.641609 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.641621 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.641743 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.641987 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.641990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.641997 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:45.642125 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.642130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.642832 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.642837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.643465 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.643473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.643978 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.643981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.644085 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.644126 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.644435 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.644445 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.644510 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.644580 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":100,"threads_started":1}
I20260808 15:51:45.653080 1426 raft_consensus.cc:1261] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Refusing update from remote peer 99a1939cc70945178b0f4455b5b0d574: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:45.653129 1613 raft_consensus.cc:1261] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Refusing update from remote peer 99a1939cc70945178b0f4455b5b0d574: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:45.653549 1728 consensus_queue.cc:1035] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:45.653781 1732 consensus_queue.cc:1035] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:45.682181 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.678508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:53833 (local address 127.30.76.65:36307)
0808 15:51:45.678615 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.678620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.678658 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:51:45.678701 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.678704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.678749 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.678824 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.679039 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.679850 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.681144 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.681356 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.681437 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.681765 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.681830 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.681880 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.682014 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.682058 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:51:45.682222 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.678211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:36307 (local address 127.30.76.67:53833)
0808 15:51:45.678517 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.678531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.678608 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.678825 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.678827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.678832 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:45.678927 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.678931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.679988 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.679991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.681016 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.681028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.681482 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.681485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.681580 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.681633 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.681913 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.681922 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.681997 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.682076 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260808 15:51:45.711465 1020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:45.711725 1020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:45.714582 1020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Sending DeleteTablet for 3 replicas of tablet d808ceeb32924917a739c3b996082fe7
I20260808 15:51:45.715477 1396 tablet_service.cc:1514] Processing DeleteTablet for tablet d808ceeb32924917a739c3b996082fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59210
I20260808 15:51:45.715515 1591 tablet_service.cc:1514] Processing DeleteTablet for tablet d808ceeb32924917a739c3b996082fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:53222
I20260808 15:51:45.716286 1209 tablet_service.cc:1514] Processing DeleteTablet for tablet d808ceeb32924917a739c3b996082fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:45146
I20260808 15:51:45.716730 1020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:51:45.716866 1780 tablet_replica.cc:330] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: stopping tablet replica
W20260808 15:51:45.717130 1020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:45.717530 1780 raft_consensus.cc:2229] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:51:45.717976 1780 raft_consensus.cc:2258] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:45.718250 1778 tablet_replica.cc:330] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: stopping tablet replica
I20260808 15:51:45.718925 1778 raft_consensus.cc:2229] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:45.719727 1778 raft_consensus.cc:2258] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:45.720432 1779 tablet_replica.cc:330] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: stopping tablet replica
I20260808 15:51:45.720714 1779 raft_consensus.cc:2229] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:45.720875 1779 raft_consensus.cc:2258] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:45.721977 1778 ts_tablet_manager.cc:1905] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:45.722800 1209 tablet_service.cc:1467] Processing CreateTablet for tablet 336d8907fdc446a496f79d521c349e15 (DEFAULT_TABLE table=test-table [id=62a9f436a13b47909b032fb295747b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:45.722815 1591 tablet_service.cc:1467] Processing CreateTablet for tablet 336d8907fdc446a496f79d521c349e15 (DEFAULT_TABLE table=test-table [id=62a9f436a13b47909b032fb295747b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:45.723080 1591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 336d8907fdc446a496f79d521c349e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.723083 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 336d8907fdc446a496f79d521c349e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.725550 1713 tablet_bootstrap.cc:492] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Bootstrap starting.
I20260808 15:51:45.725951 1714 tablet_bootstrap.cc:492] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Bootstrap starting.
I20260808 15:51:45.726539 1713 tablet_bootstrap.cc:654] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.726933 1714 tablet_bootstrap.cc:654] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.727574 1779 ts_tablet_manager.cc:1905] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:45.727721 1780 ts_tablet_manager.cc:1905] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:45.730600 1780 ts_tablet_manager.cc:1918] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:45.722491 1396 tablet_service.cc:1467] Processing CreateTablet for tablet 336d8907fdc446a496f79d521c349e15 (DEFAULT_TABLE table=test-table [id=62a9f436a13b47909b032fb295747b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:45.730924 1396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 336d8907fdc446a496f79d521c349e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:45.731014 1779 ts_tablet_manager.cc:1918] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:45.731099 1779 log.cc:1196] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/d808ceeb32924917a739c3b996082fe7
I20260808 15:51:45.731393 1779 ts_tablet_manager.cc:1939] T d808ceeb32924917a739c3b996082fe7 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting consensus metadata
I20260808 15:51:45.733513 1002 catalog_manager.cc:4841] TS 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645): tablet d808ceeb32924917a739c3b996082fe7 (table test-table [id=e3b960f0c79f4fa08e3dfb528cc205eb]) successfully deleted
I20260808 15:51:45.733619 1715 tablet_bootstrap.cc:492] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Bootstrap starting.
I20260808 15:51:45.734081 1713 tablet_bootstrap.cc:492] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: No bootstrap required, opened a new log
I20260808 15:51:45.734153 1713 ts_tablet_manager.cc:1397] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 15:51:45.734553 1715 tablet_bootstrap.cc:654] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:45.734542 1713 raft_consensus.cc:348] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.734658 1713 raft_consensus.cc:374] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.734691 1713 raft_consensus.cc:729] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e30f339dde483eba4cb174a0c1eb4d, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.734800 1713 consensus_queue.cc:260] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.735037 1713 ts_tablet_manager.cc:1428] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:45.737972 1778 ts_tablet_manager.cc:1918] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:45.738073 1778 log.cc:1196] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/d808ceeb32924917a739c3b996082fe7
I20260808 15:51:45.738399 1778 ts_tablet_manager.cc:1939] T d808ceeb32924917a739c3b996082fe7 P 05687940ea4343158a5c3d43e3109519: Deleting consensus metadata
I20260808 15:51:45.739475 1780 log.cc:1196] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/d808ceeb32924917a739c3b996082fe7
I20260808 15:51:45.739581 1003 catalog_manager.cc:4841] TS 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625): tablet d808ceeb32924917a739c3b996082fe7 (table test-table [id=e3b960f0c79f4fa08e3dfb528cc205eb]) successfully deleted
I20260808 15:51:45.740103 1780 ts_tablet_manager.cc:1939] T d808ceeb32924917a739c3b996082fe7 P 99a1939cc70945178b0f4455b5b0d574: Deleting consensus metadata
I20260808 15:51:45.741220 1003 catalog_manager.cc:4841] TS 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307): tablet d808ceeb32924917a739c3b996082fe7 (table test-table [id=e3b960f0c79f4fa08e3dfb528cc205eb]) successfully deleted
I20260808 15:51:45.741313 1714 tablet_bootstrap.cc:492] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: No bootstrap required, opened a new log
I20260808 15:51:45.741387 1714 ts_tablet_manager.cc:1397] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260808 15:51:45.741842 1714 raft_consensus.cc:348] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.741969 1714 raft_consensus.cc:374] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.742002 1714 raft_consensus.cc:729] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a1939cc70945178b0f4455b5b0d574, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.742168 1715 tablet_bootstrap.cc:492] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: No bootstrap required, opened a new log
I20260808 15:51:45.742123 1714 consensus_queue.cc:260] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.742239 1715 ts_tablet_manager.cc:1397] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 15:51:45.742383 1714 ts_tablet_manager.cc:1428] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:45.742650 1715 raft_consensus.cc:348] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.742751 1715 raft_consensus.cc:374] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:45.742779 1715 raft_consensus.cc:729] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Initialized, Role: FOLLOWER
I20260808 15:51:45.742879 1715 consensus_queue.cc:260] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.743112 1715 ts_tablet_manager.cc:1428] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:45.807457 1723 raft_consensus.cc:482] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:45.807688 1723 raft_consensus.cc:504] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.808710 1723 leader_election.cc:288] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:45.813192 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.808940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38645 (local address 127.30.76.66:47119)
0808 15:51:45.809372 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.809385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.809509 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.809856 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.809860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.809882 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.810056 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.810066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.811047 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.811050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.811976 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.811986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.812709 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.812713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.812827 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.812853 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.812936 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.813009 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":79,"thread_start_us":152,"threads_started":1}
I20260808 15:51:45.813190 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.809087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:47119 (local address 127.30.76.67:38645)
0808 15:51:45.809252 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.809257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.809454 (+ 197us) server_negotiation.cc:407] Connection header received
0808 15:51:45.809646 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.809651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.809735 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.809806 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.810197 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.810920 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.812102 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.812568 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.812687 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.812742 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.812969 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.813051 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:51:45.813769 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336d8907fdc446a496f79d521c349e15" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d" is_pre_election: true
I20260808 15:51:45.813930 1613 raft_consensus.cc:2454] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 0.
I20260808 15:51:45.813988 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.809347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:46935 (local address 127.30.76.65:36307)
0808 15:51:45.809457 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.809462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.809476 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:51:45.809573 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.809576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.809623 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.809712 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:51:45.810065 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.810853 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.812475 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.813007 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.813119 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.813163 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.813809 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.813879 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:51:45.814306 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.808914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:36307 (local address 127.30.76.66:46935)
0808 15:51:45.809353 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.809368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.809480 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.809740 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.809743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.809764 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:51:45.809932 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.809940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.810979 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.810982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.812327 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.812338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.813228 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.813232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.813679 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.813700 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.813793 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.813861 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":34,"thread_start_us":171,"threads_started":1}
I20260808 15:51:45.814468 1337 leader_election.cc:302] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 74e30f339dde483eba4cb174a0c1eb4d; no voters:
I20260808 15:51:45.814772 1723 raft_consensus.cc:2781] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:45.814862 1723 raft_consensus.cc:482] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:45.814849 1234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336d8907fdc446a496f79d521c349e15" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574" is_pre_election: true
I20260808 15:51:45.814915 1723 raft_consensus.cc:3037] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.814983 1234 raft_consensus.cc:2454] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 0.
I20260808 15:51:45.816313 1723 raft_consensus.cc:504] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.816704 1723 leader_election.cc:288] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 election: Requested vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:45.817015 1234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336d8907fdc446a496f79d521c349e15" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574"
I20260808 15:51:45.817144 1234 raft_consensus.cc:3037] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.817145 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336d8907fdc446a496f79d521c349e15" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d"
I20260808 15:51:45.817265 1613 raft_consensus.cc:3037] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:45.818130 1234 raft_consensus.cc:2454] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 1.
I20260808 15:51:45.818408 1338 leader_election.cc:302] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:45.818460 1613 raft_consensus.cc:2454] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 1.
I20260808 15:51:45.818599 1723 raft_consensus.cc:2781] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:45.818924 1723 raft_consensus.cc:686] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 1 LEADER]: Becoming Leader. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Running, Role: LEADER
I20260808 15:51:45.819208 1723 consensus_queue.cc:237] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:45.821713 1020 catalog_manager.cc:5495] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05687940ea4343158a5c3d43e3109519 (127.30.76.66). New cstate: current_term: 1 leader_uuid: "05687940ea4343158a5c3d43e3109519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } health_report { overall_health: UNKNOWN } } }
I20260808 15:51:45.838616 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.834518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.30.76.66:44625)
0808 15:51:45.834644 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.834648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.834664 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:51:45.834827 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.834830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.834907 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.835023 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.835246 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.836018 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.836913 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.837250 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.837377 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.837624 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.838243 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.838374 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.838412 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.838472 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:51:45.850775 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.847364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56771 (local address 127.30.76.67:38645)
0808 15:51:45.847584 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.847591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.847652 (+ 61us) server_negotiation.cc:407] Connection header received
0808 15:51:45.847851 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.847855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.847948 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.848037 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.848316 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.848824 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.849694 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.849926 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.850033 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.850334 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.850405 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.850485 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.850573 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.850623 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260808 15:51:45.850811 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.847240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38645 (local address 127.30.76.66:56771)
0808 15:51:45.847556 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.847572 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.847741 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.848049 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.848052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.848060 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:45.848207 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.848212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.848935 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.848940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.849571 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.849579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.850045 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.850049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.850147 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.850189 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.850489 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.850498 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.850565 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.850653 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":116,"threads_started":1}
I20260808 15:51:45.857887 1234 raft_consensus.cc:1261] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Refusing update from remote peer 05687940ea4343158a5c3d43e3109519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:45.857971 1613 raft_consensus.cc:1261] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Refusing update from remote peer 05687940ea4343158a5c3d43e3109519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:45.858383 1786 consensus_queue.cc:1035] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:45.858544 1723 consensus_queue.cc:1035] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:45.879585 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.875570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:44625 (local address 127.30.76.67:42601)
0808 15:51:45.875819 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:45.875835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:45.875915 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:45.876304 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:45.876307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:45.876316 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:51:45.876442 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.876449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.877405 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.877410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.878180 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:45.878190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.878644 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.878647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:45.878735 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.878768 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.879221 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.879227 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:51:45.879310 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:51:45.879403 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 15:51:45.879956 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:45.875763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:42601 (local address 127.30.76.66:44625)
0808 15:51:45.875866 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:45.875870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:45.875884 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:51:45.876050 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:45.876053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:45.876110 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:45.876188 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:45.876580 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.877292 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.878301 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:45.878528 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:45.878641 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:45.878874 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:45.878954 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:45.879170 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:45.879786 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 15:51:45.879840 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 15:51:45.880558 1753 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 15:51:47.447908 1018 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:47.448057 1018 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:47.450600 1018 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Sending DeleteTablet for 3 replicas of tablet 336d8907fdc446a496f79d521c349e15
I20260808 15:51:47.451156 1209 tablet_service.cc:1514] Processing DeleteTablet for tablet 336d8907fdc446a496f79d521c349e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:47 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:45146
I20260808 15:51:47.451156 1396 tablet_service.cc:1514] Processing DeleteTablet for tablet 336d8907fdc446a496f79d521c349e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:47 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59210
I20260808 15:51:47.451159 1591 tablet_service.cc:1514] Processing DeleteTablet for tablet 336d8907fdc446a496f79d521c349e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:47 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:53222
I20260808 15:51:47.451495 1829 tablet_replica.cc:330] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: stopping tablet replica
I20260808 15:51:47.451516 1830 tablet_replica.cc:330] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: stopping tablet replica
I20260808 15:51:47.451654 1829 raft_consensus.cc:2229] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:47.451661 1830 raft_consensus.cc:2229] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:47.451766 1830 raft_consensus.cc:2258] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:47.451805 1829 raft_consensus.cc:2258] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:47.451987 1826 tablet_replica.cc:330] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: stopping tablet replica
I20260808 15:51:47.452102 1826 raft_consensus.cc:2229] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:51:47.452406 1826 raft_consensus.cc:2258] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:47.452447 1829 ts_tablet_manager.cc:1905] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:47.452924 1830 ts_tablet_manager.cc:1905] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:47.455281 1829 ts_tablet_manager.cc:1918] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:51:47.455363 1829 log.cc:1196] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/336d8907fdc446a496f79d521c349e15
I20260808 15:51:47.455746 1830 ts_tablet_manager.cc:1918] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:51:47.455861 1830 log.cc:1196] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/336d8907fdc446a496f79d521c349e15
I20260808 15:51:47.456077 1826 ts_tablet_manager.cc:1905] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:47.456205 1830 ts_tablet_manager.cc:1939] T 336d8907fdc446a496f79d521c349e15 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting consensus metadata
I20260808 15:51:47.457095 1829 ts_tablet_manager.cc:1939] T 336d8907fdc446a496f79d521c349e15 P 99a1939cc70945178b0f4455b5b0d574: Deleting consensus metadata
I20260808 15:51:47.457065 1002 catalog_manager.cc:4841] TS 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645): tablet 336d8907fdc446a496f79d521c349e15 (table test-table [id=62a9f436a13b47909b032fb295747b82]) successfully deleted
I20260808 15:51:47.457975 1003 catalog_manager.cc:4841] TS 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307): tablet 336d8907fdc446a496f79d521c349e15 (table test-table [id=62a9f436a13b47909b032fb295747b82]) successfully deleted
I20260808 15:51:47.458665 1826 ts_tablet_manager.cc:1918] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:51:47.458763 1826 log.cc:1196] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/336d8907fdc446a496f79d521c349e15
I20260808 15:51:47.459044 1826 ts_tablet_manager.cc:1939] T 336d8907fdc446a496f79d521c349e15 P 05687940ea4343158a5c3d43e3109519: Deleting consensus metadata
I20260808 15:51:47.459669 1003 catalog_manager.cc:4841] TS 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625): tablet 336d8907fdc446a496f79d521c349e15 (table test-table [id=62a9f436a13b47909b032fb295747b82]) successfully deleted
Aug 08 15:51:47 dist-test-slave-sb2z krb5kdc[964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:51:47 dist-test-slave-sb2z krb5kdc[964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:47.486048 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:47.473615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.30.76.126:41073)
0808 15:51:47.473908 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:47.473915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:47.473930 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:51:47.474204 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:47.474207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:47.474251 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:47.474329 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:47.475011 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:47.475632 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:47.476559 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:47.476783 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:47.476863 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:47.483142 (+ 6279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:47.483145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:47.483147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:47.483171 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:47.484590 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:47.484957 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:47.484960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:47.484961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:47.485038 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:47.485267 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:47.485269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:47.485270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:47.485439 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:47.485536 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:47.485725 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 15:51:47.485833 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":111,"threads_started":1}
I20260808 15:51:47.488427 1018 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:51:47.488857 1018 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:47.493317 1209 tablet_service.cc:1467] Processing CreateTablet for tablet 32e28a01fdc74bf690f47ef176e45fe0 (DEFAULT_TABLE table=test-table [id=d5b4c535342b4897ae13a72cd3bfab04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:47.493357 1396 tablet_service.cc:1467] Processing CreateTablet for tablet 32e28a01fdc74bf690f47ef176e45fe0 (DEFAULT_TABLE table=test-table [id=d5b4c535342b4897ae13a72cd3bfab04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:47.493438 1591 tablet_service.cc:1467] Processing CreateTablet for tablet 32e28a01fdc74bf690f47ef176e45fe0 (DEFAULT_TABLE table=test-table [id=d5b4c535342b4897ae13a72cd3bfab04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:47.493664 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32e28a01fdc74bf690f47ef176e45fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:47.493691 1396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32e28a01fdc74bf690f47ef176e45fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:47.493687 1591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32e28a01fdc74bf690f47ef176e45fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:47.495909 1840 tablet_bootstrap.cc:492] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Bootstrap starting.
I20260808 15:51:47.496243 1846 tablet_bootstrap.cc:492] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Bootstrap starting.
I20260808 15:51:47.496452 1843 tablet_bootstrap.cc:492] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Bootstrap starting.
I20260808 15:51:47.497080 1840 tablet_bootstrap.cc:654] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:47.497202 1846 tablet_bootstrap.cc:654] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:47.497309 1843 tablet_bootstrap.cc:654] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:47.498431 1840 tablet_bootstrap.cc:492] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: No bootstrap required, opened a new log
I20260808 15:51:47.498445 1843 tablet_bootstrap.cc:492] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: No bootstrap required, opened a new log
I20260808 15:51:47.498466 1846 tablet_bootstrap.cc:492] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: No bootstrap required, opened a new log
I20260808 15:51:47.498526 1843 ts_tablet_manager.cc:1397] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:47.498526 1840 ts_tablet_manager.cc:1397] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:51:47.498564 1846 ts_tablet_manager.cc:1397] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:47.498921 1843 raft_consensus.cc:348] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.498921 1840 raft_consensus.cc:348] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.498962 1846 raft_consensus.cc:348] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.499038 1843 raft_consensus.cc:374] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:47.499038 1840 raft_consensus.cc:374] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:47.499071 1846 raft_consensus.cc:374] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:47.499074 1843 raft_consensus.cc:729] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e30f339dde483eba4cb174a0c1eb4d, State: Initialized, Role: FOLLOWER
I20260808 15:51:47.499074 1840 raft_consensus.cc:729] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a1939cc70945178b0f4455b5b0d574, State: Initialized, Role: FOLLOWER
I20260808 15:51:47.499114 1846 raft_consensus.cc:729] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Initialized, Role: FOLLOWER
I20260808 15:51:47.499248 1846 consensus_queue.cc:260] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.499233 1840 consensus_queue.cc:260] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.499233 1843 consensus_queue.cc:260] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.499567 1846 ts_tablet_manager.cc:1428] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:47.499657 1840 ts_tablet_manager.cc:1428] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 15:51:47.499657 1843 ts_tablet_manager.cc:1428] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:47.578255 1813 raft_consensus.cc:482] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:47.578390 1813 raft_consensus.cc:504] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.578852 1813 leader_election.cc:288] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:47.579267 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e28a01fdc74bf690f47ef176e45fe0" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d" is_pre_election: true
I20260808 15:51:47.579265 1234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e28a01fdc74bf690f47ef176e45fe0" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574" is_pre_election: true
I20260808 15:51:47.579399 1613 raft_consensus.cc:2454] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 0.
I20260808 15:51:47.579447 1234 raft_consensus.cc:2454] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 0.
I20260808 15:51:47.579787 1337 leader_election.cc:302] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 74e30f339dde483eba4cb174a0c1eb4d; no voters:
I20260808 15:51:47.579978 1813 raft_consensus.cc:2781] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:47.580040 1813 raft_consensus.cc:482] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:47.580067 1813 raft_consensus.cc:3037] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:47.581108 1813 raft_consensus.cc:504] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.581468 1813 leader_election.cc:288] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 election: Requested vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:47.581786 1234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e28a01fdc74bf690f47ef176e45fe0" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574"
I20260808 15:51:47.581830 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e28a01fdc74bf690f47ef176e45fe0" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d"
I20260808 15:51:47.581898 1234 raft_consensus.cc:3037] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:47.581965 1613 raft_consensus.cc:3037] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:47.582790 1234 raft_consensus.cc:2454] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 1.
I20260808 15:51:47.582949 1613 raft_consensus.cc:2454] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 1.
I20260808 15:51:47.583078 1338 leader_election.cc:302] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:47.583278 1813 raft_consensus.cc:2781] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:47.583411 1813 raft_consensus.cc:686] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 1 LEADER]: Becoming Leader. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Running, Role: LEADER
I20260808 15:51:47.583582 1813 consensus_queue.cc:237] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.585328 1020 catalog_manager.cc:5495] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05687940ea4343158a5c3d43e3109519 (127.30.76.66). New cstate: current_term: 1 leader_uuid: "05687940ea4343158a5c3d43e3109519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } health_report { overall_health: UNKNOWN } } }
I20260808 15:51:47.605491 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:47.602302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.30.76.66:44625)
0808 15:51:47.602577 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:47.602586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:47.602603 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:51:47.602639 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:47.602641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:47.602678 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:47.602766 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:51:47.603026 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:47.603614 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:47.604435 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:47.604670 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:47.604777 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:47.605020 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:51:47.605094 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:51:47.605170 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:47.605287 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 15:51:47.605338 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260808 15:51:47.611002 1613 raft_consensus.cc:1261] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Refusing update from remote peer 05687940ea4343158a5c3d43e3109519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:47.611238 1234 raft_consensus.cc:1261] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Refusing update from remote peer 05687940ea4343158a5c3d43e3109519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:47.611586 1813 consensus_queue.cc:1035] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:47.611743 1723 consensus_queue.cc:1035] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:47.642493 1017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:47.642678 1017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:47.644591 1017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Sending DeleteTablet for 3 replicas of tablet 32e28a01fdc74bf690f47ef176e45fe0
I20260808 15:51:47.645141 1209 tablet_service.cc:1514] Processing DeleteTablet for tablet 32e28a01fdc74bf690f47ef176e45fe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:47 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:45146
I20260808 15:51:47.645172 1396 tablet_service.cc:1514] Processing DeleteTablet for tablet 32e28a01fdc74bf690f47ef176e45fe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:47 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59210
I20260808 15:51:47.645231 1591 tablet_service.cc:1514] Processing DeleteTablet for tablet 32e28a01fdc74bf690f47ef176e45fe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:47 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:53222
I20260808 15:51:47.645329 1829 tablet_replica.cc:330] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: stopping tablet replica
I20260808 15:51:47.645344 1826 tablet_replica.cc:330] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: stopping tablet replica
I20260808 15:51:47.645426 1830 tablet_replica.cc:330] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: stopping tablet replica
I20260808 15:51:47.645437 1826 raft_consensus.cc:2229] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:51:47.645442 1829 raft_consensus.cc:2229] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:47.645543 1829 raft_consensus.cc:2258] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:47.645533 1830 raft_consensus.cc:2229] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:47.645565 1826 raft_consensus.cc:2258] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:47.645630 1830 raft_consensus.cc:2258] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:47.646260 1829 ts_tablet_manager.cc:1905] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:47.646318 1826 ts_tablet_manager.cc:1905] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:47.646672 1830 ts_tablet_manager.cc:1905] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:47.648782 1830 ts_tablet_manager.cc:1918] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:47.648873 1830 log.cc:1196] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/32e28a01fdc74bf690f47ef176e45fe0
I20260808 15:51:47.648859 1829 ts_tablet_manager.cc:1918] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:47.648859 1826 ts_tablet_manager.cc:1918] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:51:47.648933 1826 log.cc:1196] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/32e28a01fdc74bf690f47ef176e45fe0
I20260808 15:51:47.648933 1829 log.cc:1196] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/32e28a01fdc74bf690f47ef176e45fe0
I20260808 15:51:47.649135 1830 ts_tablet_manager.cc:1939] T 32e28a01fdc74bf690f47ef176e45fe0 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting consensus metadata
I20260808 15:51:47.649225 1826 ts_tablet_manager.cc:1939] T 32e28a01fdc74bf690f47ef176e45fe0 P 05687940ea4343158a5c3d43e3109519: Deleting consensus metadata
I20260808 15:51:47.649221 1829 ts_tablet_manager.cc:1939] T 32e28a01fdc74bf690f47ef176e45fe0 P 99a1939cc70945178b0f4455b5b0d574: Deleting consensus metadata
I20260808 15:51:47.649850 1003 catalog_manager.cc:4841] TS 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625): tablet 32e28a01fdc74bf690f47ef176e45fe0 (table test-table [id=d5b4c535342b4897ae13a72cd3bfab04]) successfully deleted
I20260808 15:51:47.649889 1002 catalog_manager.cc:4841] TS 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645): tablet 32e28a01fdc74bf690f47ef176e45fe0 (table test-table [id=d5b4c535342b4897ae13a72cd3bfab04]) successfully deleted
I20260808 15:51:47.650017 1003 catalog_manager.cc:4841] TS 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307): tablet 32e28a01fdc74bf690f47ef176e45fe0 (table test-table [id=d5b4c535342b4897ae13a72cd3bfab04]) successfully deleted
I20260808 15:51:47.650192 1017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:51:47.650592 1017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:51:47.654845 1396 tablet_service.cc:1467] Processing CreateTablet for tablet f5393bf35a3d4e72858efa0703b10fd8 (DEFAULT_TABLE table=test-table [id=2bfb6f0ebfdf4bca92e365c668316858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:47.654886 1591 tablet_service.cc:1467] Processing CreateTablet for tablet f5393bf35a3d4e72858efa0703b10fd8 (DEFAULT_TABLE table=test-table [id=2bfb6f0ebfdf4bca92e365c668316858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:47.655095 1591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5393bf35a3d4e72858efa0703b10fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:47.655095 1396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5393bf35a3d4e72858efa0703b10fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:47.657238 1846 tablet_bootstrap.cc:492] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Bootstrap starting.
I20260808 15:51:47.657300 1843 tablet_bootstrap.cc:492] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Bootstrap starting.
I20260808 15:51:47.657400 1209 tablet_service.cc:1467] Processing CreateTablet for tablet f5393bf35a3d4e72858efa0703b10fd8 (DEFAULT_TABLE table=test-table [id=2bfb6f0ebfdf4bca92e365c668316858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:51:47.657619 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5393bf35a3d4e72858efa0703b10fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:47.657992 1846 tablet_bootstrap.cc:654] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:47.658082 1843 tablet_bootstrap.cc:654] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:47.659147 1843 tablet_bootstrap.cc:492] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: No bootstrap required, opened a new log
I20260808 15:51:47.659224 1843 ts_tablet_manager.cc:1397] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:47.659457 1846 tablet_bootstrap.cc:492] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: No bootstrap required, opened a new log
I20260808 15:51:47.659642 1846 ts_tablet_manager.cc:1397] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:47.659595 1843 raft_consensus.cc:348] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.659693 1843 raft_consensus.cc:374] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:47.659720 1843 raft_consensus.cc:729] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e30f339dde483eba4cb174a0c1eb4d, State: Initialized, Role: FOLLOWER
I20260808 15:51:47.659824 1843 consensus_queue.cc:260] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.660003 1840 tablet_bootstrap.cc:492] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574: Bootstrap starting.
I20260808 15:51:47.660060 1843 ts_tablet_manager.cc:1428] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:51:47.660079 1846 raft_consensus.cc:348] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.660187 1846 raft_consensus.cc:374] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:47.660218 1846 raft_consensus.cc:729] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Initialized, Role: FOLLOWER
I20260808 15:51:47.660315 1846 consensus_queue.cc:260] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.660558 1846 ts_tablet_manager.cc:1428] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 15:51:47.660794 1840 tablet_bootstrap.cc:654] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:47.662091 1840 tablet_bootstrap.cc:492] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574: No bootstrap required, opened a new log
I20260808 15:51:47.662191 1840 ts_tablet_manager.cc:1397] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:47.662508 1840 raft_consensus.cc:348] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.662611 1840 raft_consensus.cc:374] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:47.662664 1840 raft_consensus.cc:729] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a1939cc70945178b0f4455b5b0d574, State: Initialized, Role: FOLLOWER
I20260808 15:51:47.662801 1840 consensus_queue.cc:260] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.663051 1840 ts_tablet_manager.cc:1428] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 15:51:47.681746 1813 raft_consensus.cc:482] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:51:47.681880 1813 raft_consensus.cc:504] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.682318 1813 leader_election.cc:288] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:47.682677 1232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5393bf35a3d4e72858efa0703b10fd8" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574" is_pre_election: true
I20260808 15:51:47.682670 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5393bf35a3d4e72858efa0703b10fd8" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d" is_pre_election: true
I20260808 15:51:47.682809 1613 raft_consensus.cc:2454] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 0.
I20260808 15:51:47.682811 1232 raft_consensus.cc:2454] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 0.
I20260808 15:51:47.683079 1337 leader_election.cc:302] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 74e30f339dde483eba4cb174a0c1eb4d; no voters:
I20260808 15:51:47.683279 1813 raft_consensus.cc:2781] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:51:47.683341 1813 raft_consensus.cc:482] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:51:47.683368 1813 raft_consensus.cc:3037] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:47.684300 1813 raft_consensus.cc:504] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.684588 1813 leader_election.cc:288] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 election: Requested vote from peers 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307), 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645)
I20260808 15:51:47.684906 1232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5393bf35a3d4e72858efa0703b10fd8" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1939cc70945178b0f4455b5b0d574"
I20260808 15:51:47.684921 1613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5393bf35a3d4e72858efa0703b10fd8" candidate_uuid: "05687940ea4343158a5c3d43e3109519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e30f339dde483eba4cb174a0c1eb4d"
I20260808 15:51:47.685029 1232 raft_consensus.cc:3037] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:47.685037 1613 raft_consensus.cc:3037] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:47.685878 1232 raft_consensus.cc:2454] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 1.
I20260808 15:51:47.685891 1613 raft_consensus.cc:2454] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05687940ea4343158a5c3d43e3109519 in term 1.
I20260808 15:51:47.686182 1338 leader_election.cc:302] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05687940ea4343158a5c3d43e3109519, 99a1939cc70945178b0f4455b5b0d574; no voters:
I20260808 15:51:47.686347 1813 raft_consensus.cc:2781] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:47.686441 1813 raft_consensus.cc:686] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 1 LEADER]: Becoming Leader. State: Replica: 05687940ea4343158a5c3d43e3109519, State: Running, Role: LEADER
I20260808 15:51:47.686573 1813 consensus_queue.cc:237] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } }
I20260808 15:51:47.688119 1020 catalog_manager.cc:5495] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05687940ea4343158a5c3d43e3109519 (127.30.76.66). New cstate: current_term: 1 leader_uuid: "05687940ea4343158a5c3d43e3109519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05687940ea4343158a5c3d43e3109519" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 44625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 } health_report { overall_health: UNKNOWN } } }
I20260808 15:51:47.707603 1613 raft_consensus.cc:1261] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Refusing update from remote peer 05687940ea4343158a5c3d43e3109519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:47.707792 1232 raft_consensus.cc:1261] T f5393bf35a3d4e72858efa0703b10fd8 P 99a1939cc70945178b0f4455b5b0d574 [term 1 FOLLOWER]: Refusing update from remote peer 05687940ea4343158a5c3d43e3109519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:51:47.708012 1813 consensus_queue.cc:1035] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e30f339dde483eba4cb174a0c1eb4d" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:51:47.708194 1723 consensus_queue.cc:1035] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a1939cc70945178b0f4455b5b0d574" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:51:47.721714 1801 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 15:51:49.269932 1017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:49.270079 1017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:51:49.272392 1017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a92f78b35f4edda783bf152e6cf147: Sending DeleteTablet for 3 replicas of tablet f5393bf35a3d4e72858efa0703b10fd8
I20260808 15:51:49.272888 1209 tablet_service.cc:1514] Processing DeleteTablet for tablet f5393bf35a3d4e72858efa0703b10fd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:49 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:45146
I20260808 15:51:49.272998 1591 tablet_service.cc:1514] Processing DeleteTablet for tablet f5393bf35a3d4e72858efa0703b10fd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:49 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:53222
I20260808 15:51:49.273037 1396 tablet_service.cc:1514] Processing DeleteTablet for tablet f5393bf35a3d4e72858efa0703b10fd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:51:49 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59210
I20260808 15:51:49.273358 1895 tablet_replica.cc:330] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: stopping tablet replica
I20260808 15:51:49.273356 1894 tablet_replica.cc:330] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: stopping tablet replica
I20260808 15:51:49.273386 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 1119
I20260808 15:51:49.277534 1895 raft_consensus.cc:2229] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:51:49.277920 1895 raft_consensus.cc:2258] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:49.279321 1895 ts_tablet_manager.cc:1905] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:49.282119 1894 raft_consensus.cc:2229] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:51:49.282269 1894 raft_consensus.cc:2258] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:51:49.283118 1894 ts_tablet_manager.cc:1905] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:51:49.283756 1895 ts_tablet_manager.cc:1918] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:51:49.283921 1895 log.cc:1196] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/f5393bf35a3d4e72858efa0703b10fd8
I20260808 15:51:49.284405 1895 ts_tablet_manager.cc:1939] T f5393bf35a3d4e72858efa0703b10fd8 P 05687940ea4343158a5c3d43e3109519: Deleting consensus metadata
I20260808 15:51:49.286005 1894 ts_tablet_manager.cc:1918] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:51:49.286083 1894 log.cc:1196] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/f5393bf35a3d4e72858efa0703b10fd8
I20260808 15:51:49.286329 1894 ts_tablet_manager.cc:1939] T f5393bf35a3d4e72858efa0703b10fd8 P 74e30f339dde483eba4cb174a0c1eb4d: Deleting consensus metadata
I20260808 15:51:49.287129 1002 catalog_manager.cc:4841] TS 74e30f339dde483eba4cb174a0c1eb4d (127.30.76.67:38645): tablet f5393bf35a3d4e72858efa0703b10fd8 (table test-table [id=2bfb6f0ebfdf4bca92e365c668316858]) successfully deleted
I20260808 15:51:49.294054 1003 catalog_manager.cc:4841] TS 05687940ea4343158a5c3d43e3109519 (127.30.76.66:44625): tablet f5393bf35a3d4e72858efa0703b10fd8 (table test-table [id=2bfb6f0ebfdf4bca92e365c668316858]) successfully deleted
W20260808 15:51:49.301298 1003 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:36307 (error 108)
I20260808 15:51:49.301589 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 1312
I20260808 15:51:49.309734 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:49.304087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:36307 (local address 127.0.0.1:45170)
0808 15:51:49.309562 (+ 5475us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:49.309640 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5394,"thread_start_us":507,"threads_started":1}
W20260808 15:51:49.309882 1003 catalog_manager.cc:4584] TS 99a1939cc70945178b0f4455b5b0d574 (127.30.76.65:36307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5393bf35a3d4e72858efa0703b10fd8: Network error: Client connection negotiation failed: client connection to 127.30.76.65:36307: connect: Connection refused (error 111)
I20260808 15:51:49.315932 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 1503
I20260808 15:51:49.330394 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 980
2026-08-08T15:51:49Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6510 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:49 dist-test-slave-sb2z krb5kdc[1900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:49 dist-test-slave-sb2z krb5kdc[1900](info): set up 2 sockets
Aug 08 15:51:49 dist-test-slave-sb2z krb5kdc[1900](info): commencing operation
krb5kdc: starting...
W20260808 15:51:50.587515 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:50 dist-test-slave-sb2z krb5kdc[1900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:50.710058 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:44669
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:44669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:50.828609 1919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:50.828897 1919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:50.828969 1919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:50.832966 1919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:50.833062 1919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:50.833101 1919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:50.833153 1919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:50.833192 1919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:50.838351 1919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:44669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:44669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:50.839380 1919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:50.840412 1919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:50.849401 1930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:50.849442 1926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:50.849447 1925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:50.849491 1919 server_base.cc:1034] running on GCE node
I20260808 15:51:50.850270 1919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:50.851339 1919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:50.852523 1919 hybrid_clock.cc:648] HybridClock initialized: now 1786204310852528 us; error 52 us; skew 500 ppm
Aug 08 15:51:50 dist-test-slave-sb2z krb5kdc[1900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204310, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:50.855223 1919 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:50.856108 1919 webserver.cc:460] Webserver started at http://127.30.76.126:40423/ using document root <none> and password file <none>
I20260808 15:51:50.856652 1919 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:50.856712 1919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:50.856876 1919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:50.858801 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "c32f8268d35045b6bb945ae7823ea165"
format_stamp: "Formatted at 2026-08-08 15:51:50 on dist-test-slave-sb2z"
server_key: "16c3a24e444c829c76ebd9a6c322c6d8"
server_key_iv: "5ef61f485a9f51fc7b32ef4580c6eaf9"
server_key_version: "encryptionkey@0"
I20260808 15:51:50.859299 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "c32f8268d35045b6bb945ae7823ea165"
format_stamp: "Formatted at 2026-08-08 15:51:50 on dist-test-slave-sb2z"
server_key: "16c3a24e444c829c76ebd9a6c322c6d8"
server_key_iv: "5ef61f485a9f51fc7b32ef4580c6eaf9"
server_key_version: "encryptionkey@0"
I20260808 15:51:50.862841 1919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:51:50.865140 1942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:50.866313 1919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:50.866422 1919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "c32f8268d35045b6bb945ae7823ea165"
format_stamp: "Formatted at 2026-08-08 15:51:50 on dist-test-slave-sb2z"
server_key: "16c3a24e444c829c76ebd9a6c322c6d8"
server_key_iv: "5ef61f485a9f51fc7b32ef4580c6eaf9"
server_key_version: "encryptionkey@0"
I20260808 15:51:50.866503 1919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:50.876412 1919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:50.878927 1919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:50.879065 1919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:50.885865 1919 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:44669
I20260808 15:51:50.885877 2011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:44669 every 8 connection(s)
I20260808 15:51:50.886858 1919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:50.889678 2012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:50.895130 2012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165: Bootstrap starting.
I20260808 15:51:50.896908 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 1919
I20260808 15:51:50.897001 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:50.897274 31025 external_mini_cluster.cc:1428] Setting key 3ce988646e66a8b65cc1f38ce908ecf2
I20260808 15:51:50.897778 2012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:50.898710 2012 log.cc:824] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:50.900866 2012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165: No bootstrap required, opened a new log
I20260808 15:51:50.905503 2012 raft_consensus.cc:348] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32f8268d35045b6bb945ae7823ea165" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44669 } }
I20260808 15:51:50.905838 2012 raft_consensus.cc:374] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:50.905912 2012 raft_consensus.cc:729] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32f8268d35045b6bb945ae7823ea165, State: Initialized, Role: FOLLOWER
I20260808 15:51:50.906492 2012 consensus_queue.cc:260] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32f8268d35045b6bb945ae7823ea165" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44669 } }
I20260808 15:51:50.906680 2012 raft_consensus.cc:388] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:50.906776 2012 raft_consensus.cc:482] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:50.906883 2012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 15:51:50 dist-test-slave-sb2z krb5kdc[1900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:50.908208 2012 raft_consensus.cc:504] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32f8268d35045b6bb945ae7823ea165" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44669 } }
I20260808 15:51:50.908576 2012 leader_election.cc:302] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c32f8268d35045b6bb945ae7823ea165; no voters:
I20260808 15:51:50.909525 2012 leader_election.cc:288] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:50.910745 2021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:50.911170 2021 raft_consensus.cc:686] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [term 1 LEADER]: Becoming Leader. State: Replica: c32f8268d35045b6bb945ae7823ea165, State: Running, Role: LEADER
I20260808 15:51:50.911464 2021 consensus_queue.cc:237] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32f8268d35045b6bb945ae7823ea165" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44669 } }
I20260808 15:51:50.912290 2012 sys_catalog.cc:564] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:50.913470 2023 sys_catalog.cc:455] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c32f8268d35045b6bb945ae7823ea165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32f8268d35045b6bb945ae7823ea165" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44669 } } }
I20260808 15:51:50.913601 2023 sys_catalog.cc:458] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:50.913970 2022 sys_catalog.cc:455] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c32f8268d35045b6bb945ae7823ea165. Latest consensus state: current_term: 1 leader_uuid: "c32f8268d35045b6bb945ae7823ea165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32f8268d35045b6bb945ae7823ea165" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44669 } } }
I20260808 15:51:50.914072 2022 sys_catalog.cc:458] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:50.914986 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:50.898399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.30.76.126:44669)
0808 15:51:50.898810 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:50.898829 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:50.898872 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:51:50.899718 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:50.899739 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:50.900052 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:50.900366 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:50.901311 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:50.902847 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:50.904191 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:50.904920 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:50.905095 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:50.908321 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:50.908329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:50.908381 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:50.908413 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:50.910578 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:50.911318 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:50.911324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:50.911330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:50.911477 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:50.911971 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:50.911977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:50.911982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:50.912682 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:50.912895 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:50.913461 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 15:51:50.913803 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
I20260808 15:51:50.914165 2032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:50.918846 2032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:50.923974 2032 catalog_manager.cc:1347] Generated new cluster ID: 43330c97bbfc4106a4325d2542336091
I20260808 15:51:50.924038 2032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:50.938606 2032 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:50.939874 2032 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:50.946795 2032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c32f8268d35045b6bb945ae7823ea165: Generated new TSK 0
I20260808 15:51:50.947484 2032 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:50.980429 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44669
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:51.094115 2048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:51.094378 2048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:51.094455 2048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:51.098670 2048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:51.098776 2048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:51.098868 2048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:51.104097 2048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:51.105125 2048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:51.106160 2048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:51.115849 2057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:51.115855 2053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:51.115855 2054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:51.116137 2048 server_base.cc:1034] running on GCE node
I20260808 15:51:51.116575 2048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:51.117226 2048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:51.118425 2048 hybrid_clock.cc:648] HybridClock initialized: now 1786204311118405 us; error 35 us; skew 500 ppm
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[1900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:51.121321 2048 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:51.122097 2048 webserver.cc:460] Webserver started at http://127.30.76.65:40475/ using document root <none> and password file <none>
I20260808 15:51:51.122640 2048 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:51.122726 2048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:51.122942 2048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:51.124913 2048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "106938f4fcf846fcb7bf253f7f515aac"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "f2d099b9b25785965bf1c645df5fa449"
server_key_iv: "083a14c9d933f912c804feb5e4d09ec5"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.125465 2048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "106938f4fcf846fcb7bf253f7f515aac"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "f2d099b9b25785965bf1c645df5fa449"
server_key_iv: "083a14c9d933f912c804feb5e4d09ec5"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.129006 2048 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:51:51.131445 2067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.132614 2048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:51.132746 2048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "106938f4fcf846fcb7bf253f7f515aac"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "f2d099b9b25785965bf1c645df5fa449"
server_key_iv: "083a14c9d933f912c804feb5e4d09ec5"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.132853 2048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:51.151696 2048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:51.154520 2048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:51.154722 2048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:51.155318 2048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:51.156209 2048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:51.156277 2048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.156348 2048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:51.156394 2048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.165203 2048 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:40313
I20260808 15:51:51.165233 2224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:40313 every 8 connection(s)
I20260808 15:51:51.166085 2048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:51.167169 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 2048
I20260808 15:51:51.167358 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:51.167708 31025 external_mini_cluster.cc:1428] Setting key d8fab393987dafbc71dbec6ff5758e63
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[1900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:51:51.180459 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:51.168096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:37649 (local address 127.30.76.126:44669)
0808 15:51:51.168244 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:51.168249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:51.169099 (+ 850us) server_negotiation.cc:407] Connection header received
0808 15:51:51.170595 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:51.170601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:51.170663 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:51.170751 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:51.171920 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.172763 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.173851 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.174091 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.174209 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:51.176920 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:51.176922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:51.176924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:51.176948 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:51.178414 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:51.179039 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:51.179042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:51.179043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:51.179145 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:51.179439 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:51.179441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:51.179442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:51.179710 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:51.179810 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:51.180236 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 15:51:51.180339 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:51:51.181334 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:51.168502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:44669 (local address 127.30.76.65:37649)
0808 15:51:51.168955 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:51.169006 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:51.170403 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:51.170906 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:51.170912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:51.171257 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:51.171782 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:51.171797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.172871 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.172873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:51.173725 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:51.173733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.174248 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.174252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:51.174414 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:51.175003 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:51.175019 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:51.176760 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:51.178530 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:51.178537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:51.178551 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:51.178941 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:51.179226 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:51.179228 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:51.179230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:51.179361 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:51.179812 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:51.179817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:51.180024 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 15:51:51.180190 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 15:51:51.180419 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":133,"threads_started":1}
I20260808 15:51:51.182543 2225 heartbeater.cc:347] Connected to a master server at 127.30.76.126:44669
I20260808 15:51:51.182811 2225 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:51.183326 2225 heartbeater.cc:511] Master 127.30.76.126:44669 requested a full tablet report, sending...
I20260808 15:51:51.185050 1965 ts_manager.cc:194] Registered new tserver with Master: 106938f4fcf846fcb7bf253f7f515aac (127.30.76.65:40313)
I20260808 15:51:51.186748 1965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:37649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:51.196161 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44669
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:51.307479 2230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:51.307752 2230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:51.307794 2230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:51.312088 2230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:51.312145 2230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:51.312202 2230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:51.318851 2230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:51.320310 2230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:51.321713 2230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:51.331663 2237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:51.331739 2239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:51.331812 2230 server_base.cc:1034] running on GCE node
W20260808 15:51:51.331739 2236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:51.332326 2230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:51.332942 2230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:51.334128 2230 hybrid_clock.cc:648] HybridClock initialized: now 1786204311334107 us; error 41 us; skew 500 ppm
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[1900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:51.337024 2230 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:51.337818 2230 webserver.cc:460] Webserver started at http://127.30.76.66:34399/ using document root <none> and password file <none>
I20260808 15:51:51.338315 2230 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:51.338371 2230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:51.338527 2230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:51.340499 2230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "9dcb4e213c2942f4ae4dd91560cae4e3"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "7bc78aca3eaddd7b7271355635ce893a"
server_key_iv: "d62701c2fcec195751fd9d9c52bcb62f"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.341027 2230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "9dcb4e213c2942f4ae4dd91560cae4e3"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "7bc78aca3eaddd7b7271355635ce893a"
server_key_iv: "d62701c2fcec195751fd9d9c52bcb62f"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.344544 2230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:51.346870 2253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.348073 2230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:51.348168 2230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "9dcb4e213c2942f4ae4dd91560cae4e3"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "7bc78aca3eaddd7b7271355635ce893a"
server_key_iv: "d62701c2fcec195751fd9d9c52bcb62f"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.348248 2230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:51.377281 2230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:51.380524 2230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:51.380690 2230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:51.381259 2230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:51.382036 2230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:51.382083 2230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.382125 2230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:51.382146 2230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.390559 2230 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:46307
I20260808 15:51:51.390587 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:46307 every 8 connection(s)
I20260808 15:51:51.391422 2230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:51.392689 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 2230
I20260808 15:51:51.392799 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:51.393129 31025 external_mini_cluster.cc:1428] Setting key 51eda0e01487f751585b1f7c1fe4a310
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[1900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:51:51.405326 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:51.393246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:50801 (local address 127.30.76.126:44669)
0808 15:51:51.393364 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:51.393368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:51.394292 (+ 924us) server_negotiation.cc:407] Connection header received
0808 15:51:51.395740 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:51.395746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:51.395810 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:51.395900 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:51.397005 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.397855 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.398960 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.399190 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.399293 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:51.401741 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:51.401745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:51.401747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:51.401771 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:51.403227 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:51.403803 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:51.403807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:51.403808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:51.403888 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:51.404235 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:51.404238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:51.404240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:51.404436 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:51.404541 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:51.405021 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 15:51:51.405149 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 15:51:51.406081 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:51.393680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:44669 (local address 127.30.76.66:50801)
0808 15:51:51.394132 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:51.394194 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:51.395474 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:51.396047 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:51.396052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:51.396409 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:51.396858 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:51.396871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.397980 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.397983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:51.398824 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:51.398834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.399341 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.399345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:51.399561 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:51.400169 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:51.400185 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:51.401574 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:51.403344 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:51.403350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:51.403364 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:51.403703 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:51.403986 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:51.403989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:51.403991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:51.404129 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:51.404541 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:51.404548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:51.404786 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 15:51:51.404961 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 15:51:51.405284 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":135,"threads_started":1}
I20260808 15:51:51.407220 2413 heartbeater.cc:347] Connected to a master server at 127.30.76.126:44669
I20260808 15:51:51.407471 2413 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:51.407984 2413 heartbeater.cc:511] Master 127.30.76.126:44669 requested a full tablet report, sending...
I20260808 15:51:51.408972 1965 ts_manager.cc:194] Registered new tserver with Master: 9dcb4e213c2942f4ae4dd91560cae4e3 (127.30.76.66:46307)
I20260808 15:51:51.409526 1965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:50801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:51.420809 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44669
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:51.534610 2419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:51.534855 2419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:51.534895 2419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:51.539692 2419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:51.539770 2419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:51.539880 2419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:51.545205 2419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:51.546176 2419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:51.547447 2419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:51.557090 2430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:51.557102 2427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:51.557112 2428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:51.557508 2419 server_base.cc:1034] running on GCE node
I20260808 15:51:51.557860 2419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:51.558383 2419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:51.559571 2419 hybrid_clock.cc:648] HybridClock initialized: now 1786204311559541 us; error 38 us; skew 500 ppm
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[1900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:51.562243 2419 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:51.563001 2419 webserver.cc:460] Webserver started at http://127.30.76.67:37017/ using document root <none> and password file <none>
I20260808 15:51:51.563515 2419 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:51.563643 2419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:51.563822 2419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:51.565726 2419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "940a5d53175642f383464370c623d9c0"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "af025bb28e56eb6e3bd8b4555c556b56"
server_key_iv: "e17c708563b81f83dc357eca7d23cfc2"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.566286 2419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "940a5d53175642f383464370c623d9c0"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "af025bb28e56eb6e3bd8b4555c556b56"
server_key_iv: "e17c708563b81f83dc357eca7d23cfc2"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.570010 2419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:51:51.572398 2439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.573513 2419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:51.573643 2419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "940a5d53175642f383464370c623d9c0"
format_stamp: "Formatted at 2026-08-08 15:51:51 on dist-test-slave-sb2z"
server_key: "af025bb28e56eb6e3bd8b4555c556b56"
server_key_iv: "e17c708563b81f83dc357eca7d23cfc2"
server_key_version: "encryptionkey@0"
I20260808 15:51:51.573745 2419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:51.589007 2419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:51.592123 2419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:51.592312 2419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:51.592867 2419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:51.593631 2419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:51.593705 2419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.593772 2419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:51.593811 2419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:51.602403 2419 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:46481
I20260808 15:51:51.602425 2604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:46481 every 8 connection(s)
I20260808 15:51:51.603271 2419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:51.607704 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 2419
I20260808 15:51:51.607836 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:51.608124 31025 external_mini_cluster.cc:1428] Setting key 85287198a47cc14411f29e7f767f417c
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[1900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:51.616895 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:51.604956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:34741 (local address 127.30.76.126:44669)
0808 15:51:51.605116 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:51.605121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:51.605924 (+ 803us) server_negotiation.cc:407] Connection header received
0808 15:51:51.607157 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:51.607160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:51.607208 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:51.607285 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:51.608527 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.609326 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.610713 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.610920 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.611032 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:51.613456 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:51.613462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:51.613464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:51.613492 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:51.614919 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:51.615424 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:51.615428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:51.615429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:51.615515 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:51.615868 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:51.615872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:51.615874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:51.616055 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:51.616159 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:51.616615 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 15:51:51.616727 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:51:51.617633 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:51.605359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:44669 (local address 127.30.76.67:34741)
0808 15:51:51.605781 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:51.605820 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:51.606958 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:51.607449 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:51.607455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:51.607789 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:51.608348 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:51.608362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.609458 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.609461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:51.610606 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:51.610615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:51.611068 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:51.611071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:51.611206 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:51.611793 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:51.611810 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:51.613307 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:51.615045 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:51.615051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:51.615065 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:51.615319 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:51.615634 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:51.615637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:51.615639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:51.615773 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:51.616159 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:51.616166 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:51.616388 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 15:51:51.616571 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 15:51:51.616859 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260808 15:51:51.618625 2605 heartbeater.cc:347] Connected to a master server at 127.30.76.126:44669
I20260808 15:51:51.618850 2605 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:51.619297 2605 heartbeater.cc:511] Master 127.30.76.126:44669 requested a full tablet report, sending...
I20260808 15:51:51.620303 1965 ts_manager.cc:194] Registered new tserver with Master: 940a5d53175642f383464370c623d9c0 (127.30.76.67:46481)
I20260808 15:51:51.620934 1965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:34741
I20260808 15:51:51.621953 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 15:51:51.637190 2017 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:51:51.635796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.30.76.126:44669)
0808 15:51:51.635930 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:51.635936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:51.636139 (+ 203us) server_negotiation.cc:407] Connection header received
0808 15:51:51.636273 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:51.636275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:51.636314 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:51.636393 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:51.637029 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57854: BlockingRecv error: recv got EOF from 127.0.0.1:57854 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 15:51:51.638268 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 2048
I20260808 15:51:51.648586 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 2230
I20260808 15:51:51.661242 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 2419
I20260808 15:51:51.674356 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 1919
2026-08-08T15:51:51Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2341 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[2625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[2625](info): set up 2 sockets
Aug 08 15:51:51 dist-test-slave-sb2z krb5kdc[2625](info): commencing operation
krb5kdc: starting...
W20260808 15:51:52.909360 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:52 dist-test-slave-sb2z krb5kdc[2625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:53.032106 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:37411
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:37411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:53.145432 2642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:53.145711 2642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:53.145752 2642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:53.149839 2642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:53.149902 2642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:53.149921 2642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:53.149938 2642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:53.149953 2642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:53.155143 2642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:37411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:37411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:53.156215 2642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:53.157219 2642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:53.165654 2648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:53.165663 2647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:53.165690 2653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:53.165903 2642 server_base.cc:1034] running on GCE node
I20260808 15:51:53.166571 2642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:53.167748 2642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:53.168928 2642 hybrid_clock.cc:648] HybridClock initialized: now 1786204313168919 us; error 33 us; skew 500 ppm
Aug 08 15:51:53 dist-test-slave-sb2z krb5kdc[2625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204313, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:53.174568 2642 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:53.175469 2642 webserver.cc:460] Webserver started at http://127.30.76.126:46453/ using document root <none> and password file <none>
I20260808 15:51:53.176070 2642 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:53.176160 2642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:53.176391 2642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:53.178447 2642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "4f6f8704fae34c1ea8c325c149e4d171"
format_stamp: "Formatted at 2026-08-08 15:51:53 on dist-test-slave-sb2z"
server_key: "95f56faba561619081249755770b7007"
server_key_iv: "2ec3d787cac4f41337c4bb4a5af19d63"
server_key_version: "encryptionkey@0"
I20260808 15:51:53.179026 2642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "4f6f8704fae34c1ea8c325c149e4d171"
format_stamp: "Formatted at 2026-08-08 15:51:53 on dist-test-slave-sb2z"
server_key: "95f56faba561619081249755770b7007"
server_key_iv: "2ec3d787cac4f41337c4bb4a5af19d63"
server_key_version: "encryptionkey@0"
I20260808 15:51:53.182708 2642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:51:53.185077 2661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:53.186275 2642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:53.186416 2642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "4f6f8704fae34c1ea8c325c149e4d171"
format_stamp: "Formatted at 2026-08-08 15:51:53 on dist-test-slave-sb2z"
server_key: "95f56faba561619081249755770b7007"
server_key_iv: "2ec3d787cac4f41337c4bb4a5af19d63"
server_key_version: "encryptionkey@0"
I20260808 15:51:53.186520 2642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:53.198038 2642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:53.201259 2642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:53.201458 2642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:53.208482 2642 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:37411
I20260808 15:51:53.208781 2732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:37411 every 8 connection(s)
I20260808 15:51:53.209472 2642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:53.212255 2733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:53.217640 2733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171: Bootstrap starting.
I20260808 15:51:53.218384 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 2642
I20260808 15:51:53.218482 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:53.218768 31025 external_mini_cluster.cc:1428] Setting key bfdf45818f4b4bbaab0ebd7f5d215a2d
I20260808 15:51:53.220580 2733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:53.221406 2733 log.cc:824] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:53.223673 2733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171: No bootstrap required, opened a new log
Aug 08 15:51:53 dist-test-slave-sb2z krb5kdc[2625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:53.228300 2733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6f8704fae34c1ea8c325c149e4d171" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37411 } }
I20260808 15:51:53.228546 2733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:53.228626 2733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f6f8704fae34c1ea8c325c149e4d171, State: Initialized, Role: FOLLOWER
I20260808 15:51:53.229110 2733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6f8704fae34c1ea8c325c149e4d171" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37411 } }
I20260808 15:51:53.229262 2733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:53.229343 2733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:53.229442 2733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:53.230341 2733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6f8704fae34c1ea8c325c149e4d171" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37411 } }
I20260808 15:51:53.230695 2733 leader_election.cc:302] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f6f8704fae34c1ea8c325c149e4d171; no voters:
I20260808 15:51:53.231068 2733 leader_election.cc:288] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:53.231145 2740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:53.231360 2740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [term 1 LEADER]: Becoming Leader. State: Replica: 4f6f8704fae34c1ea8c325c149e4d171, State: Running, Role: LEADER
I20260808 15:51:53.231703 2740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6f8704fae34c1ea8c325c149e4d171" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37411 } }
I20260808 15:51:53.232465 2733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:53.233459 2741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f6f8704fae34c1ea8c325c149e4d171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6f8704fae34c1ea8c325c149e4d171" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37411 } } }
I20260808 15:51:53.233579 2741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:53.233516 2745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f6f8704fae34c1ea8c325c149e4d171. Latest consensus state: current_term: 1 leader_uuid: "4f6f8704fae34c1ea8c325c149e4d171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6f8704fae34c1ea8c325c149e4d171" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37411 } } }
I20260808 15:51:53.233650 2745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:53.234095 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:53.219950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57630 (local address 127.30.76.126:37411)
0808 15:51:53.220379 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:53.220395 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:53.220432 (+ 37us) server_negotiation.cc:407] Connection header received
0808 15:51:53.221128 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:53.221149 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:53.221480 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:53.221842 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:53.222723 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:53.223769 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:53.224919 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:53.225376 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:53.225499 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:53.228443 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:53.228447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:53.228461 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:53.228488 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:53.230285 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:53.230763 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:53.230768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:53.230772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:53.230931 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:53.231259 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:53.231262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:53.231264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:53.231829 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:53.232014 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:53.232441 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 15:51:53.232845 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":140,"threads_started":1}
I20260808 15:51:53.237232 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 15:51:53.237739 2762 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:51:53.237825 2762 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:51:53.237917 2756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:53.238647 2756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:53.244889 2756 catalog_manager.cc:1347] Generated new cluster ID: d8a03318a0df498e8940c2a5a34161f6
I20260808 15:51:53.244954 2756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:53.260756 2756 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:53.262022 2756 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:53.262954 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:53.257424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.30.76.126:37411)
0808 15:51:53.257545 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:53.257549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:53.257735 (+ 186us) server_negotiation.cc:407] Connection header received
0808 15:51:53.257865 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:53.257868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:53.257921 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:53.258023 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:53.258596 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:53.259401 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:53.260664 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:53.260995 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:53.261122 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:53.261994 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:53.261998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:53.262001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:51:53.262486 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:53.262515 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:51:53.262531 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:53.262631 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:53.262717 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 15:51:53.262793 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":43}
I20260808 15:51:53.269224 2756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f6f8704fae34c1ea8c325c149e4d171: Generated new TSK 0
I20260808 15:51:53.269990 2756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:51:53.275113 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 2642
2026-08-08T15:51:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1602 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:53 dist-test-slave-sb2z krb5kdc[2779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:53 dist-test-slave-sb2z krb5kdc[2779](info): set up 2 sockets
Aug 08 15:51:53 dist-test-slave-sb2z krb5kdc[2779](info): commencing operation
krb5kdc: starting...
W20260808 15:51:54.488018 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:54 dist-test-slave-sb2z krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:54.610881 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:35513
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:35513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:54.720510 2799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:54.720798 2799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:54.720875 2799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:54.724820 2799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:54.724907 2799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:54.724947 2799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:54.724999 2799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:54.725040 2799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:54.730185 2799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:35513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:35513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:54.731218 2799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:54.732256 2799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:54.740801 2811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:54.740863 2799 server_base.cc:1034] running on GCE node
W20260808 15:51:54.740881 2809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:54.740979 2808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:54.741597 2799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:54.742643 2799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:54.743801 2799 hybrid_clock.cc:648] HybridClock initialized: now 1786204314743780 us; error 34 us; skew 500 ppm
Aug 08 15:51:54 dist-test-slave-sb2z krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:54.746860 2799 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:54.747834 2799 webserver.cc:460] Webserver started at http://127.30.76.126:46113/ using document root <none> and password file <none>
I20260808 15:51:54.748401 2799 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:54.748462 2799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:54.748672 2799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:54.750661 2799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "aa198ed736054b88ba4c5c57b05b0465"
format_stamp: "Formatted at 2026-08-08 15:51:54 on dist-test-slave-sb2z"
server_key: "7ba59417d2a3892f9c14edc1aa5abc94"
server_key_iv: "09ce904c5dcbc2ac4ba493e63f560042"
server_key_version: "encryptionkey@0"
I20260808 15:51:54.751243 2799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "aa198ed736054b88ba4c5c57b05b0465"
format_stamp: "Formatted at 2026-08-08 15:51:54 on dist-test-slave-sb2z"
server_key: "7ba59417d2a3892f9c14edc1aa5abc94"
server_key_iv: "09ce904c5dcbc2ac4ba493e63f560042"
server_key_version: "encryptionkey@0"
I20260808 15:51:54.754928 2799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:51:54.757417 2821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:54.758670 2799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:54.758808 2799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "aa198ed736054b88ba4c5c57b05b0465"
format_stamp: "Formatted at 2026-08-08 15:51:54 on dist-test-slave-sb2z"
server_key: "7ba59417d2a3892f9c14edc1aa5abc94"
server_key_iv: "09ce904c5dcbc2ac4ba493e63f560042"
server_key_version: "encryptionkey@0"
I20260808 15:51:54.758914 2799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:54.776851 2799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:54.780203 2799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:54.780408 2799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:54.787636 2799 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:35513
I20260808 15:51:54.787664 2890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:35513 every 8 connection(s)
I20260808 15:51:54.788555 2799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:54.791239 2891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:54.796556 2891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465: Bootstrap starting.
I20260808 15:51:54.797267 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 2799
I20260808 15:51:54.797392 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:54.797693 31025 external_mini_cluster.cc:1428] Setting key 518fbe3df889a305b63ec7eb807096be
I20260808 15:51:54.799407 2891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:54.800163 2891 log.cc:824] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:54.801898 2891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465: No bootstrap required, opened a new log
I20260808 15:51:54.805428 2891 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa198ed736054b88ba4c5c57b05b0465" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35513 } }
I20260808 15:51:54.805646 2891 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:54.805722 2891 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa198ed736054b88ba4c5c57b05b0465, State: Initialized, Role: FOLLOWER
Aug 08 15:51:54 dist-test-slave-sb2z krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:54.806188 2891 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa198ed736054b88ba4c5c57b05b0465" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35513 } }
I20260808 15:51:54.806365 2891 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:54.806437 2891 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:54.806564 2891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:54.807456 2891 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa198ed736054b88ba4c5c57b05b0465" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35513 } }
I20260808 15:51:54.807807 2891 leader_election.cc:302] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa198ed736054b88ba4c5c57b05b0465; no voters:
I20260808 15:51:54.808147 2891 leader_election.cc:288] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:54.808262 2898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:54.808506 2898 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [term 1 LEADER]: Becoming Leader. State: Replica: aa198ed736054b88ba4c5c57b05b0465, State: Running, Role: LEADER
I20260808 15:51:54.808868 2898 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa198ed736054b88ba4c5c57b05b0465" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35513 } }
I20260808 15:51:54.809501 2891 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:54.811317 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa198ed736054b88ba4c5c57b05b0465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa198ed736054b88ba4c5c57b05b0465" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35513 } } }
I20260808 15:51:54.811450 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:54.811851 2900 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa198ed736054b88ba4c5c57b05b0465. Latest consensus state: current_term: 1 leader_uuid: "aa198ed736054b88ba4c5c57b05b0465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa198ed736054b88ba4c5c57b05b0465" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35513 } } }
I20260808 15:51:54.812299 2900 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:54.812168 2918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:54.812860 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:54.798877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.30.76.126:35513)
0808 15:51:54.799312 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:54.799326 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:54.799360 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:51:54.800065 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:54.800084 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:54.800449 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:54.800766 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:54.801599 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:54.802607 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:54.803479 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:54.803988 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:54.804177 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:54.806851 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:54.806857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:54.806877 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:54.806903 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:54.809037 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:54.809582 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:54.809588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:54.809593 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:54.809731 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:54.810075 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:54.810078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:54.810080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:54.810576 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:54.810758 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:54.811412 (+ 654us) server_negotiation.cc:299] Negotiation successful
0808 15:51:54.811782 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":139,"threads_started":1}
I20260808 15:51:54.815212 2918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:54.820313 2918 catalog_manager.cc:1347] Generated new cluster ID: 6f8609ce319046f9b8a75e44d54deb3f
I20260808 15:51:54.820380 2918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:54.831311 2918 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:54.832252 2918 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:54.846936 2918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa198ed736054b88ba4c5c57b05b0465: Generated new TSK 0
I20260808 15:51:54.847726 2918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:54.882560 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:35513
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:55.004511 2927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:55.004822 2927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:55.004901 2927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:55.009366 2927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:55.009449 2927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:55.009541 2927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:55.015030 2927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:35513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:55.016170 2927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:55.017220 2927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:55.026666 2933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:55.026690 2934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:55.026674 2936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:55.027066 2927 server_base.cc:1034] running on GCE node
I20260808 15:51:55.027442 2927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:55.028070 2927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:55.029250 2927 hybrid_clock.cc:648] HybridClock initialized: now 1786204315029235 us; error 27 us; skew 500 ppm
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:55.032120 2927 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:55.032956 2927 webserver.cc:460] Webserver started at http://127.30.76.65:40437/ using document root <none> and password file <none>
I20260808 15:51:55.033509 2927 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:55.033596 2927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:55.033806 2927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:55.035933 2927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "54c22e447bfe4b0ba3434f422636ca84"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "dd55ce96a58096036b99091de0296f5d"
server_key_iv: "fe99c4895ae517d8b6a2746c83700e94"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.036504 2927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "54c22e447bfe4b0ba3434f422636ca84"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "dd55ce96a58096036b99091de0296f5d"
server_key_iv: "fe99c4895ae517d8b6a2746c83700e94"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.040251 2927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:51:55.042588 2951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.043818 2927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:55.043942 2927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "54c22e447bfe4b0ba3434f422636ca84"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "dd55ce96a58096036b99091de0296f5d"
server_key_iv: "fe99c4895ae517d8b6a2746c83700e94"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.044049 2927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:55.058912 2927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:55.062057 2927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:55.062248 2927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:55.062754 2927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:55.063623 2927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:55.063673 2927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.063712 2927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:55.063733 2927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.072407 2927 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:45783
I20260808 15:51:55.072431 3095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:45783 every 8 connection(s)
I20260808 15:51:55.073316 2927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:55.079470 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 2927
I20260808 15:51:55.079641 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:55.079928 31025 external_mini_cluster.cc:1428] Setting key f77fe4bc8faabc2941b32337ca034577
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:55.087421 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.075050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:48817 (local address 127.30.76.126:35513)
0808 15:51:55.075189 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:55.075193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:55.076092 (+ 899us) server_negotiation.cc:407] Connection header received
0808 15:51:55.077464 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:55.077467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:55.077527 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:55.077629 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:55.078738 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.079292 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.080560 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.080866 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.080985 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.083502 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:55.083505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:55.083507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:55.083567 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:55.085208 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.085707 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.085714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.085718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.085816 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.086144 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.086150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.086155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.086433 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:55.086549 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:55.087016 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 15:51:55.087183 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:51:55.088028 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.075406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:35513 (local address 127.30.76.65:48817)
0808 15:51:55.075952 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:55.075993 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:55.077259 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:55.077804 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:55.077811 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:55.078148 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:55.078586 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:55.078598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.079415 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.079419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:55.080435 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:55.080445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.081017 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.081020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:55.081253 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.081809 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:55.081825 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:55.083329 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:55.085336 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:55.085340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:55.085351 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:55.085600 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:55.085911 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:55.085913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:55.085914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:55.086039 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:55.086553 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:55.086557 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:55.086776 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 15:51:55.086928 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 15:51:55.087181 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":201,"threads_started":1}
I20260808 15:51:55.089257 3097 heartbeater.cc:347] Connected to a master server at 127.30.76.126:35513
I20260808 15:51:55.089476 3097 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:55.089943 3097 heartbeater.cc:511] Master 127.30.76.126:35513 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:51:55.091408 2842 ts_manager.cc:194] Registered new tserver with Master: 54c22e447bfe4b0ba3434f422636ca84 (127.30.76.65:45783)
I20260808 15:51:55.092536 2842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:48817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:55.107234 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:35513
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:55.216073 3102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:55.216315 3102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:55.216357 3102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:55.220643 3102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:55.220709 3102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:55.220820 3102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:55.226151 3102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:35513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:55.227173 3102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:55.228204 3102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:55.237604 3108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:55.237604 3110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:55.237668 3107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:55.237886 3102 server_base.cc:1034] running on GCE node
I20260808 15:51:55.238405 3102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:55.239017 3102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:55.240201 3102 hybrid_clock.cc:648] HybridClock initialized: now 1786204315240181 us; error 29 us; skew 500 ppm
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:55.242936 3102 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:55.243790 3102 webserver.cc:460] Webserver started at http://127.30.76.66:42307/ using document root <none> and password file <none>
I20260808 15:51:55.244302 3102 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:55.244360 3102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:55.244518 3102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:55.246423 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "3a7b91e515a4406abb2c722753595822"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "876fd3fa55e4d10821a00f07b7435206"
server_key_iv: "fbb308c83ed761c80fbfcf53ce5f69a2"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.246976 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "3a7b91e515a4406abb2c722753595822"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "876fd3fa55e4d10821a00f07b7435206"
server_key_iv: "fbb308c83ed761c80fbfcf53ce5f69a2"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.250702 3102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 15:51:55.252997 3121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.254155 3102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 15:51:55.254282 3102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3a7b91e515a4406abb2c722753595822"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "876fd3fa55e4d10821a00f07b7435206"
server_key_iv: "fbb308c83ed761c80fbfcf53ce5f69a2"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.254381 3102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:55.275274 3102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:55.278273 3102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:55.278465 3102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:55.279071 3102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:55.279923 3102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:55.279969 3102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.280035 3102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:55.280071 3102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.288455 3102 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:41161
I20260808 15:51:55.288514 3292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:41161 every 8 connection(s)
I20260808 15:51:55.289338 3102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:55.293465 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 3102
I20260808 15:51:55.293592 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:55.293918 31025 external_mini_cluster.cc:1428] Setting key ad45f9d07fcefb220b8a252d9d69782c
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:55.303555 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.291063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:36265 (local address 127.30.76.126:35513)
0808 15:51:55.291194 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:55.291198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:55.292021 (+ 823us) server_negotiation.cc:407] Connection header received
0808 15:51:55.293398 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:55.293401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:55.293451 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:55.293518 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:55.294966 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.295686 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.296804 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.297033 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.297145 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.299520 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:55.299558 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:55.299561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:55.299591 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:55.301392 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.301947 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.301952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.301954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.302064 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.302400 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.302404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.302406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.302649 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:55.302761 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:55.303215 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 15:51:55.303351 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:51:55.304555 3296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.291373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:35513 (local address 127.30.76.66:36265)
0808 15:51:55.291867 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:55.291907 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:55.293121 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:55.293718 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:55.293727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:55.294232 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:55.294827 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:55.294843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.295813 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.295817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:55.296664 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:55.296673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.297188 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.297192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:55.297358 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.297920 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:55.297936 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:55.299352 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:55.301510 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:55.301515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:55.301527 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:55.301844 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:55.302174 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:55.302176 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:55.302178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:55.302309 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:55.302750 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:55.302755 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:55.302974 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 15:51:55.303126 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 15:51:55.303456 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":184,"threads_started":1}
I20260808 15:51:55.305754 3294 heartbeater.cc:347] Connected to a master server at 127.30.76.126:35513
I20260808 15:51:55.305956 3294 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:55.306408 3294 heartbeater.cc:511] Master 127.30.76.126:35513 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:51:55.307443 2842 ts_manager.cc:194] Registered new tserver with Master: 3a7b91e515a4406abb2c722753595822 (127.30.76.66:41161)
I20260808 15:51:55.308007 2842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:36265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:55.323066 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:35513
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:55.434808 3301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:55.435084 3301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:55.435176 3301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:55.439142 3301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:55.439229 3301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:55.439316 3301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:55.444514 3301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:35513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:55.445552 3301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:55.446766 3301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:55.456223 3309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:55.456208 3313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:55.456211 3311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:55.456596 3301 server_base.cc:1034] running on GCE node
I20260808 15:51:55.456951 3301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:55.457557 3301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:55.458736 3301 hybrid_clock.cc:648] HybridClock initialized: now 1786204315458720 us; error 28 us; skew 500 ppm
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:55.461472 3301 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:55.462360 3301 webserver.cc:460] Webserver started at http://127.30.76.67:46505/ using document root <none> and password file <none>
I20260808 15:51:55.462980 3301 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:55.463068 3301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:55.463302 3301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:55.465380 3301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "f98f35d737d743f2a202d520ef8d23cf"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "f7bb78daa1c2865688130282ae4b48b2"
server_key_iv: "b6772b1ca11782c0fd2f9b77187d80f3"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.465929 3301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "f98f35d737d743f2a202d520ef8d23cf"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "f7bb78daa1c2865688130282ae4b48b2"
server_key_iv: "b6772b1ca11782c0fd2f9b77187d80f3"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.469609 3301 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 15:51:55.471906 3327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.472998 3301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:55.473140 3301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "f98f35d737d743f2a202d520ef8d23cf"
format_stamp: "Formatted at 2026-08-08 15:51:55 on dist-test-slave-sb2z"
server_key: "f7bb78daa1c2865688130282ae4b48b2"
server_key_iv: "b6772b1ca11782c0fd2f9b77187d80f3"
server_key_version: "encryptionkey@0"
I20260808 15:51:55.473245 3301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:55.528383 3301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:55.531795 3301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:55.532006 3301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:55.532631 3301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:55.533493 3301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:55.533562 3301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.533635 3301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:55.533676 3301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:55.542490 3301 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:37109
I20260808 15:51:55.542515 3494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:37109 every 8 connection(s)
I20260808 15:51:55.543411 3301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:55.550024 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 3301
I20260808 15:51:55.550153 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:55.550450 31025 external_mini_cluster.cc:1428] Setting key dd9152f08be8ac7ca23928a884616298
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:55.557192 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.545250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:40585 (local address 127.30.76.126:35513)
0808 15:51:55.545416 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:55.545420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:55.546238 (+ 818us) server_negotiation.cc:407] Connection header received
0808 15:51:55.547695 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:55.547698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:55.547749 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:55.547816 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:55.549031 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.549557 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.550811 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.551028 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.551150 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.553905 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:55.553908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:55.553910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:55.553934 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:55.555376 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.555885 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.555887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.555888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.555966 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.556266 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.556269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.556270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.556440 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:55.556513 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:55.556967 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 15:51:55.557064 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:51:55.558046 3498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.545639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:35513 (local address 127.30.76.67:40585)
0808 15:51:55.546085 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:55.546129 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:55.547419 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:55.547973 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:55.547979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:55.548345 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:55.548900 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:55.548915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.549670 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.549673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:55.550678 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:55.550687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.551186 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.551190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:55.551381 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.552085 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:55.552105 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:55.553748 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:55.555474 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:55.555478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:55.555492 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:55.555793 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:55.556039 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:55.556041 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:55.556043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:55.556181 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:55.556525 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:55.556530 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:55.556757 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 15:51:55.556904 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:51:55.557162 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":104,"threads_started":1}
I20260808 15:51:55.559197 3495 heartbeater.cc:347] Connected to a master server at 127.30.76.126:35513
I20260808 15:51:55.559408 3495 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:55.559911 3495 heartbeater.cc:511] Master 127.30.76.126:35513 requested a full tablet report, sending...
I20260808 15:51:55.560863 2842 ts_manager.cc:194] Registered new tserver with Master: f98f35d737d743f2a202d520ef8d23cf (127.30.76.67:37109)
I20260808 15:51:55.561391 2842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:40585
I20260808 15:51:55.564404 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204315, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:55.591316 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:55.583093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.30.76.126:35513)
0808 15:51:55.583274 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:55.583279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:55.583408 (+ 129us) server_negotiation.cc:407] Connection header received
0808 15:51:55.583592 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:55.583596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:55.583651 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:55.583741 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:55.584458 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.584979 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.585818 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:55.586034 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:55.586142 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:55.588673 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:55.588677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:55.588679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:55.588703 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:55.589909 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.590292 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.590295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.590296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.590369 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:55.590585 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:55.590587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:55.590588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:55.590806 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:55.590911 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:55.591096 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 15:51:55.591193 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 15:51:55.591773 2842 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40948
I20260808 15:51:55.592623 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 2927
I20260808 15:51:55.603400 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 3102
I20260808 15:51:55.614163 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 3301
I20260808 15:51:55.627188 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 2799
2026-08-08T15:51:55Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2350 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[3510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[3510](info): set up 2 sockets
Aug 08 15:51:55 dist-test-slave-sb2z krb5kdc[3510](info): commencing operation
krb5kdc: starting...
W20260808 15:51:56.843185 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:56 dist-test-slave-sb2z krb5kdc[3510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:56.964529 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41747
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:57.076696 3524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:57.076932 3524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:57.076970 3524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:57.081080 3524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:57.081131 3524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:57.081156 3524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:57.081172 3524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:57.081220 3524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:57.086329 3524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:57.087255 3524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:57.088256 3524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:57.096807 3533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.096879 3532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.096875 3535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:57.097054 3524 server_base.cc:1034] running on GCE node
I20260808 15:51:57.097654 3524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:57.098626 3524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:57.099783 3524 hybrid_clock.cc:648] HybridClock initialized: now 1786204317099773 us; error 26 us; skew 500 ppm
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:57.102420 3524 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:57.103262 3524 webserver.cc:460] Webserver started at http://127.30.76.126:44157/ using document root <none> and password file <none>
I20260808 15:51:57.103870 3524 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:57.103956 3524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:57.104188 3524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:57.106140 3524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "f67ebab240c6490fa423bab67334f187"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "29b6e5a94d3d2b2369b4db70d1db4c75"
server_key_iv: "3e663ade1b17d5706381d6d35c0d0fc7"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.106688 3524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "f67ebab240c6490fa423bab67334f187"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "29b6e5a94d3d2b2369b4db70d1db4c75"
server_key_iv: "3e663ade1b17d5706381d6d35c0d0fc7"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.110280 3524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 15:51:57.112568 3545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.113720 3524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:51:57.113852 3524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "f67ebab240c6490fa423bab67334f187"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "29b6e5a94d3d2b2369b4db70d1db4c75"
server_key_iv: "3e663ade1b17d5706381d6d35c0d0fc7"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.113950 3524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:57.126449 3524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:57.129295 3524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:57.129482 3524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:57.136204 3524 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41747
I20260808 15:51:57.136231 3628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41747 every 8 connection(s)
I20260808 15:51:57.137077 3524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:57.139669 3633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:57.140480 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 3524
I20260808 15:51:57.140606 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:57.140887 31025 external_mini_cluster.cc:1428] Setting key 039ccf8367170109439ef15afbf1665f
I20260808 15:51:57.145459 3633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187: Bootstrap starting.
I20260808 15:51:57.147668 3633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:57.148408 3633 log.cc:824] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:57.150511 3633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187: No bootstrap required, opened a new log
I20260808 15:51:57.154506 3633 raft_consensus.cc:348] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67ebab240c6490fa423bab67334f187" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41747 } }
I20260808 15:51:57.154649 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.142238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56632 (local address 127.30.76.126:41747)
0808 15:51:57.142657 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:57.142671 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:57.142704 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:51:57.143310 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:57.143326 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:57.143703 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:57.144012 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:57.144812 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.145774 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.146585 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.147045 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.147227 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.149927 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:57.149932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:57.149944 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:57.149969 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:57.151833 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.152395 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.152400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.152404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.152511 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.152812 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.152814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.152815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.153215 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:57.153365 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:57.153654 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 15:51:57.153947 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260808 15:51:57.154738 3633 raft_consensus.cc:374] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:57.154807 3633 raft_consensus.cc:729] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67ebab240c6490fa423bab67334f187, State: Initialized, Role: FOLLOWER
I20260808 15:51:57.155202 3633 consensus_queue.cc:260] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67ebab240c6490fa423bab67334f187" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41747 } }
I20260808 15:51:57.155316 3633 raft_consensus.cc:388] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:57.155400 3633 raft_consensus.cc:482] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:57.155496 3633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:57.156533 3633 raft_consensus.cc:504] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67ebab240c6490fa423bab67334f187" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41747 } }
I20260808 15:51:57.156869 3633 leader_election.cc:302] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f67ebab240c6490fa423bab67334f187; no voters:
I20260808 15:51:57.157166 3633 leader_election.cc:288] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:57.157240 3638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:57.157481 3638 raft_consensus.cc:686] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [term 1 LEADER]: Becoming Leader. State: Replica: f67ebab240c6490fa423bab67334f187, State: Running, Role: LEADER
I20260808 15:51:57.157758 3638 consensus_queue.cc:237] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67ebab240c6490fa423bab67334f187" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41747 } }
I20260808 15:51:57.158288 3633 sys_catalog.cc:564] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:57.159394 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f67ebab240c6490fa423bab67334f187. Latest consensus state: current_term: 1 leader_uuid: "f67ebab240c6490fa423bab67334f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67ebab240c6490fa423bab67334f187" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41747 } } }
I20260808 15:51:57.159391 3639 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f67ebab240c6490fa423bab67334f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67ebab240c6490fa423bab67334f187" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41747 } } }
I20260808 15:51:57.159574 3639 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:57.159574 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187 [sys.catalog]: This master's current role is: LEADER
W20260808 15:51:57.163015 3658 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:51:57.163105 3658 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:51:57.163204 3647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:57.163967 3647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:57.168619 3647 catalog_manager.cc:1347] Generated new cluster ID: 28efa4ba99714e2f8a78bec1acecfa24
I20260808 15:51:57.168699 3647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:57.180387 3647 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:57.181619 3647 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:57.186446 3647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f67ebab240c6490fa423bab67334f187: Generated new TSK 0
I20260808 15:51:57.187086 3647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:57.223811 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41747
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:57.338202 3664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:57.338439 3664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:57.338539 3664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:57.342608 3664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:57.342664 3664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:57.342753 3664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:57.347860 3664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:57.348845 3664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:57.349897 3664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:57.359373 3671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.359413 3676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.359475 3670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:57.359740 3664 server_base.cc:1034] running on GCE node
I20260808 15:51:57.360185 3664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:57.360814 3664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:57.361992 3664 hybrid_clock.cc:648] HybridClock initialized: now 1786204317361978 us; error 26 us; skew 500 ppm
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:57.364679 3664 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:57.365489 3664 webserver.cc:460] Webserver started at http://127.30.76.65:44923/ using document root <none> and password file <none>
I20260808 15:51:57.366034 3664 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:57.366122 3664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:57.366350 3664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:57.368351 3664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "8b8e11e84cad4020b0346b0d08e06ecf"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "5e595eaba1410277a2dc3fa9e5637761"
server_key_iv: "ebfee77e011ac9c9d8f4fb28f4cd8b08"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.368903 3664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "8b8e11e84cad4020b0346b0d08e06ecf"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "5e595eaba1410277a2dc3fa9e5637761"
server_key_iv: "ebfee77e011ac9c9d8f4fb28f4cd8b08"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.372525 3664 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:57.374760 3685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.375908 3664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:57.376045 3664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "8b8e11e84cad4020b0346b0d08e06ecf"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "5e595eaba1410277a2dc3fa9e5637761"
server_key_iv: "ebfee77e011ac9c9d8f4fb28f4cd8b08"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.376158 3664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:57.396347 3664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:57.399005 3664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:57.399197 3664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:57.399813 3664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:57.400601 3664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:57.400671 3664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.400739 3664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:57.400784 3664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.409683 3664 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:46221
I20260808 15:51:57.409726 3851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:46221 every 8 connection(s)
I20260808 15:51:57.410523 3664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:57.420405 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 3664
I20260808 15:51:57.420536 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:57.420854 31025 external_mini_cluster.cc:1428] Setting key 747374818b6b285d88f61583cf495d4b
I20260808 15:51:57.424377 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.412254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:47319 (local address 127.30.76.126:41747)
0808 15:51:57.412427 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:57.412431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:57.413195 (+ 764us) server_negotiation.cc:407] Connection header received
0808 15:51:57.414553 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:57.414557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:57.414617 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:57.414723 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:57.415861 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.416400 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.417634 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.417884 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.417992 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.420502 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:57.420508 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:57.420511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:57.420540 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:57.422209 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.422802 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.422804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.422806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.422883 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.423290 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.423292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.423293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.423455 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:57.423574 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:57.424168 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 15:51:57.424265 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 15:51:57.425339 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.412632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41747 (local address 127.30.76.65:47319)
0808 15:51:57.413053 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:57.413093 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:57.414343 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:57.414890 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:57.414896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:57.415235 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:57.415708 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:57.415721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.416524 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.416528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:57.417511 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:57.417521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.418048 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.418052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:57.418235 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.418814 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:57.418831 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:57.420337 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:57.422328 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:57.422333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:57.422344 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:57.422705 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:57.422980 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:57.422986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:57.422991 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:57.423191 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:57.423575 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:57.423585 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:57.423919 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 15:51:57.424145 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 15:51:57.424492 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260808 15:51:57.426559 3853 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41747
I20260808 15:51:57.426808 3853 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:57.427387 3853 heartbeater.cc:511] Master 127.30.76.126:41747 requested a full tablet report, sending...
I20260808 15:51:57.429000 3570 ts_manager.cc:194] Registered new tserver with Master: 8b8e11e84cad4020b0346b0d08e06ecf (127.30.76.65:46221)
I20260808 15:51:57.430184 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:47319
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:57.448271 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41747
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:57.565439 3858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:57.565677 3858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:57.565769 3858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:57.569762 3858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:57.569816 3858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:57.569870 3858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:51:57.575022 3858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:57.576095 3858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:57.577085 3858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:57.587086 3870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.587078 3864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.587055 3866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:57.587287 3858 server_base.cc:1034] running on GCE node
I20260808 15:51:57.587870 3858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:57.588548 3858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:57.589743 3858 hybrid_clock.cc:648] HybridClock initialized: now 1786204317589712 us; error 45 us; skew 500 ppm
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:57.592659 3858 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:51:57.593446 3858 webserver.cc:460] Webserver started at http://127.30.76.66:37455/ using document root <none> and password file <none>
I20260808 15:51:57.594048 3858 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:57.594112 3858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:57.594283 3858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:57.596457 3858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "b047379a355b4d3982e826f96cf16383"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "399182edf6d1f38f2141824f84a02854"
server_key_iv: "76177b354abffdc94dceb7e120528105"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.596971 3858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "b047379a355b4d3982e826f96cf16383"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "399182edf6d1f38f2141824f84a02854"
server_key_iv: "76177b354abffdc94dceb7e120528105"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.600513 3858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:51:57.602789 3881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.604295 3858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:51:57.604390 3858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "b047379a355b4d3982e826f96cf16383"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "399182edf6d1f38f2141824f84a02854"
server_key_iv: "76177b354abffdc94dceb7e120528105"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.604471 3858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:57.619266 3858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:57.622391 3858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:57.622587 3858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:57.623133 3858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:57.624029 3858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:57.624076 3858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.624143 3858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:57.624200 3858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.632357 3858 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:45761
I20260808 15:51:57.632402 4046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:45761 every 8 connection(s)
I20260808 15:51:57.633229 3858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:51:57.634519 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 3858
I20260808 15:51:57.634685 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:51:57.635038 31025 external_mini_cluster.cc:1428] Setting key 13bba8c7dcfbd9a50b6ba865ae8a027e
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:51:57.648193 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.635027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:40159 (local address 127.30.76.126:41747)
0808 15:51:57.635510 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:57.635517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:57.636137 (+ 620us) server_negotiation.cc:407] Connection header received
0808 15:51:57.637593 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:57.637599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:57.637668 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:57.637752 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:57.639058 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.639647 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.641389 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.641603 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.641680 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.644582 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:57.644586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:57.644588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:57.644611 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:57.646071 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.646581 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.646585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.646587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.646681 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.647116 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.647119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.647120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.647310 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:57.647399 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:57.647947 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 15:51:57.648054 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:51:57.649230 4049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.635425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41747 (local address 127.30.76.66:40159)
0808 15:51:57.635976 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:57.636029 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:57.637390 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:57.637875 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:57.637881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:57.638215 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:57.638899 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:57.638921 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.639772 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.639775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:57.641259 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:57.641274 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.641763 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.641770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:57.641997 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.642601 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:57.642627 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:57.644410 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:57.646195 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:57.646201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:57.646215 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:57.646474 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:57.646776 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:57.646780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:57.646782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:57.646968 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:57.647414 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:57.647419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:57.647720 (+ 301us) client_negotiation.cc:769] Sending connection context
0808 15:51:57.647886 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 15:51:57.648225 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":129,"threads_started":1}
I20260808 15:51:57.650437 4047 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41747
I20260808 15:51:57.650707 4047 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:57.651199 4047 heartbeater.cc:511] Master 127.30.76.126:41747 requested a full tablet report, sending...
I20260808 15:51:57.652350 3570 ts_manager.cc:194] Registered new tserver with Master: b047379a355b4d3982e826f96cf16383 (127.30.76.66:45761)
I20260808 15:51:57.653285 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:40159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:51:57.664327 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41747
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:57.776497 4052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:57.776733 4052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:57.776814 4052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:57.781143 4052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:57.781198 4052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:57.781293 4052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:51:57.786661 4052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:57.787699 4052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:57.788697 4052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:57.797989 4061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.798012 4058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:57.797950 4064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:57.798266 4052 server_base.cc:1034] running on GCE node
I20260808 15:51:57.798704 4052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:57.799311 4052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:57.800470 4052 hybrid_clock.cc:648] HybridClock initialized: now 1786204317800449 us; error 31 us; skew 500 ppm
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:57.803189 4052 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:51:57.804060 4052 webserver.cc:460] Webserver started at http://127.30.76.67:42755/ using document root <none> and password file <none>
I20260808 15:51:57.804596 4052 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:57.804693 4052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:57.804915 4052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:57.806849 4052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "fb4c3be713a748f1bd8ba4232ec353d3"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "0cca850d71212bf458e07042803c4de1"
server_key_iv: "03ed906aa1e3f9565fae3d58c6d897a7"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.807449 4052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "fb4c3be713a748f1bd8ba4232ec353d3"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "0cca850d71212bf458e07042803c4de1"
server_key_iv: "03ed906aa1e3f9565fae3d58c6d897a7"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.811012 4052 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:51:57.813304 4073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.814416 4052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:57.814548 4052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "fb4c3be713a748f1bd8ba4232ec353d3"
format_stamp: "Formatted at 2026-08-08 15:51:57 on dist-test-slave-sb2z"
server_key: "0cca850d71212bf458e07042803c4de1"
server_key_iv: "03ed906aa1e3f9565fae3d58c6d897a7"
server_key_version: "encryptionkey@0"
I20260808 15:51:57.814656 4052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:57.834399 4052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:57.837677 4052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:57.837877 4052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:57.838479 4052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:57.839291 4052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:57.839362 4052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.839437 4052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:57.839478 4052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:57.848613 4052 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44733
I20260808 15:51:57.848639 4231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44733 every 8 connection(s)
I20260808 15:51:57.849471 4052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:51:57.850874 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 4052
I20260808 15:51:57.851020 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:51:57.851408 31025 external_mini_cluster.cc:1428] Setting key 26e0af275b0b01de72ca5a68aa1667cb
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:57.862736 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.851736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:57653 (local address 127.30.76.126:41747)
0808 15:51:57.851904 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:57.851909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:57.852582 (+ 673us) server_negotiation.cc:407] Connection header received
0808 15:51:57.854202 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:57.854205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:57.854251 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:57.854319 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:57.855404 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.855988 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.857056 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.857264 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.857350 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.859866 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:57.859869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:57.859871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:57.859893 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:57.861057 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.861507 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.861510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.861511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.861582 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.861864 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.861866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.861867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.862024 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:57.862099 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:57.862519 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 15:51:57.862607 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:51:57.863473 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.851845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41747 (local address 127.30.76.67:57653)
0808 15:51:57.852429 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:57.852484 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:57.854007 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:57.854469 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:57.854475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:57.854829 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:57.855284 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:57.855297 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.856092 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.856095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:57.856952 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:57.856961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.857398 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.857400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:57.857527 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.858098 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:57.858113 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:57.859711 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:57.861163 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:57.861168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:57.861178 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:57.861423 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:57.861659 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:57.861661 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:57.861663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:57.861787 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:57.862108 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:57.862112 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:57.862325 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 15:51:57.862472 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:51:57.862697 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":146,"threads_started":1}
I20260808 15:51:57.864621 4234 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41747
I20260808 15:51:57.864849 4234 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:57.865370 4234 heartbeater.cc:511] Master 127.30.76.126:41747 requested a full tablet report, sending...
I20260808 15:51:57.866406 3569 ts_manager.cc:194] Registered new tserver with Master: fb4c3be713a748f1bd8ba4232ec353d3 (127.30.76.67:44733)
I20260808 15:51:57.867024 3569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:57653
I20260808 15:51:57.876904 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:51:57.903728 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.894571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56696 (local address 127.30.76.65:46221)
0808 15:51:57.894876 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:57.894891 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:57.894916 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:51:57.894966 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:57.894971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:57.895160 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:57.895278 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:57.895990 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.896772 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.897567 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.897936 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.898016 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.900416 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:57.900425 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:57.900437 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:57.900462 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:57.902166 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.902530 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.902535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.902536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.902614 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.902843 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.902846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.902847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.903075 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:57.903246 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:57.903380 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 15:51:57.903505 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":109,"threads_started":1}
Aug 08 15:51:57 dist-test-slave-sb2z krb5kdc[3510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:57.914156 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:57.906574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.30.76.126:41747)
0808 15:51:57.906726 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:57.906729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:57.906744 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:51:57.906854 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:57.906856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:57.906896 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:57.906962 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:57.907570 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.908082 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.908840 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:57.909045 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:57.909123 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:57.911276 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:57.911279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:57.911281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:57.911304 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:57.912747 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.913094 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.913096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.913097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.913171 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:57.913390 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:57.913392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:57.913393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:57.913584 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:57.913693 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:57.913853 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 15:51:57.913937 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 15:51:57.914552 3569 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638
I20260808 15:51:57.915177 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 3664
I20260808 15:51:57.935652 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 3858
I20260808 15:51:57.943845 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 4052
I20260808 15:51:57.951893 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 3524
2026-08-08T15:51:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2323 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:51:58 dist-test-slave-sb2z krb5kdc[4256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:51:58 dist-test-slave-sb2z krb5kdc[4256](info): set up 2 sockets
Aug 08 15:51:58 dist-test-slave-sb2z krb5kdc[4256](info): commencing operation
krb5kdc: starting...
W20260808 15:51:59.264084 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:51:59 dist-test-slave-sb2z krb5kdc[4256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:51:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:51:59.389873 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45511
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:59.500387 4277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:59.500636 4277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:59.500677 4277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:59.504739 4277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:51:59.504789 4277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:59.504809 4277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:59.504825 4277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:51:59.504861 4277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:51:59.512521 4277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:59.513535 4277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:59.514537 4277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:59.523102 4284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:59.523111 4283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:59.523289 4277 server_base.cc:1034] running on GCE node
W20260808 15:51:59.523111 4286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:59.524080 4277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:59.525146 4277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:59.526341 4277 hybrid_clock.cc:648] HybridClock initialized: now 1786204319526327 us; error 31 us; skew 500 ppm
Aug 08 15:51:59 dist-test-slave-sb2z krb5kdc[4256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204319, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:59.529027 4277 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:51:59.529773 4277 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:59.529867 4277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:59.530099 4277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:59.532204 4277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "f33fc1b809e8421c98ff02cdc4277478"
format_stamp: "Formatted at 2026-08-08 15:51:59 on dist-test-slave-sb2z"
server_key: "bb8f5cdc997a23667dae59940dca6b1a"
server_key_iv: "526f743d5d03066f1d4f27f9f162fdd0"
server_key_version: "encryptionkey@0"
I20260808 15:51:59.532771 4277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "f33fc1b809e8421c98ff02cdc4277478"
format_stamp: "Formatted at 2026-08-08 15:51:59 on dist-test-slave-sb2z"
server_key: "bb8f5cdc997a23667dae59940dca6b1a"
server_key_iv: "526f743d5d03066f1d4f27f9f162fdd0"
server_key_version: "encryptionkey@0"
I20260808 15:51:59.536455 4277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:51:59.538857 4292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:59.540036 4277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:51:59.540184 4277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "f33fc1b809e8421c98ff02cdc4277478"
format_stamp: "Formatted at 2026-08-08 15:51:59 on dist-test-slave-sb2z"
server_key: "bb8f5cdc997a23667dae59940dca6b1a"
server_key_iv: "526f743d5d03066f1d4f27f9f162fdd0"
server_key_version: "encryptionkey@0"
I20260808 15:51:59.540292 4277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:59.556661 4277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:59.559306 4277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:59.559509 4277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:59.566468 4277 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45511
I20260808 15:51:59.566740 4365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45511 every 8 connection(s)
I20260808 15:51:59.567155 4277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:51:59.569842 4369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:51:59.575078 4369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478: Bootstrap starting.
I20260808 15:51:59.576311 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 4277
I20260808 15:51:59.576452 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:51:59.576740 31025 external_mini_cluster.cc:1428] Setting key 91a576f6b350094c578473be27e04130
I20260808 15:51:59.578058 4369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478: Neither blocks nor log segments found. Creating new log.
I20260808 15:51:59.578822 4369 log.cc:824] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478: Log is configured to *not* fsync() on all Append() calls
I20260808 15:51:59.581043 4369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478: No bootstrap required, opened a new log
Aug 08 15:51:59 dist-test-slave-sb2z krb5kdc[4256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:59.585506 4369 raft_consensus.cc:348] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33fc1b809e8421c98ff02cdc4277478" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45511 } }
I20260808 15:51:59.585759 4369 raft_consensus.cc:374] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:51:59.585839 4369 raft_consensus.cc:729] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f33fc1b809e8421c98ff02cdc4277478, State: Initialized, Role: FOLLOWER
I20260808 15:51:59.586306 4369 consensus_queue.cc:260] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33fc1b809e8421c98ff02cdc4277478" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45511 } }
I20260808 15:51:59.586426 4369 raft_consensus.cc:388] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:51:59.586505 4369 raft_consensus.cc:482] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:51:59.586609 4369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:51:59.587451 4369 raft_consensus.cc:504] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33fc1b809e8421c98ff02cdc4277478" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45511 } }
I20260808 15:51:59.587805 4369 leader_election.cc:302] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f33fc1b809e8421c98ff02cdc4277478; no voters:
I20260808 15:51:59.588153 4369 leader_election.cc:288] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:51:59.588235 4375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:51:59.588467 4375 raft_consensus.cc:686] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [term 1 LEADER]: Becoming Leader. State: Replica: f33fc1b809e8421c98ff02cdc4277478, State: Running, Role: LEADER
I20260808 15:51:59.588812 4375 consensus_queue.cc:237] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33fc1b809e8421c98ff02cdc4277478" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45511 } }
I20260808 15:51:59.589442 4369 sys_catalog.cc:564] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:51:59.590555 4376 sys_catalog.cc:455] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f33fc1b809e8421c98ff02cdc4277478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33fc1b809e8421c98ff02cdc4277478" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45511 } } }
I20260808 15:51:59.590680 4376 sys_catalog.cc:458] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:59.591068 4377 sys_catalog.cc:455] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f33fc1b809e8421c98ff02cdc4277478. Latest consensus state: current_term: 1 leader_uuid: "f33fc1b809e8421c98ff02cdc4277478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33fc1b809e8421c98ff02cdc4277478" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45511 } } }
I20260808 15:51:59.591881 4377 sys_catalog.cc:458] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478 [sys.catalog]: This master's current role is: LEADER
I20260808 15:51:59.592078 4387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:51:59.592483 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:59.577831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.30.76.126:45511)
0808 15:51:59.578252 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:59.578269 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:59.578303 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:51:59.578902 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:59.578917 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:59.579347 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:59.579716 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:59.580572 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:59.581585 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:59.582757 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:59.583215 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:59.583338 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:59.586307 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:59.586312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:59.586324 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:59.586351 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:59.588465 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:59.588979 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:59.588985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:59.588990 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:59.589105 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:59.589428 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:59.589432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:59.589434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:59.589993 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:59.590877 (+ 884us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:59.591020 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 15:51:59.591318 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":133,"threads_started":1}
I20260808 15:51:59.595494 4387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:51:59.601138 4387 catalog_manager.cc:1347] Generated new cluster ID: 6fc1e91b6d1a4d2d86caa0a932e6f4f8
I20260808 15:51:59.601209 4387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:51:59.607803 4387 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:51:59.608959 4387 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:51:59.614624 4387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478: Generated new TSK 0
I20260808 15:51:59.615242 4387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:51:59.648548 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45511
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:59.760519 4408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:59.760823 4408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:59.760898 4408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:51:59.764962 4408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:51:59.765051 4408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:51:59.765146 4408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:51:59.770416 4408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:51:59.771570 4408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:51:59.772636 4408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:51:59.782146 4419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:51:59.782145 4414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:59.782307 4408 server_base.cc:1034] running on GCE node
W20260808 15:51:59.782357 4415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:51:59.782809 4408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:51:59.783454 4408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:51:59.784647 4408 hybrid_clock.cc:648] HybridClock initialized: now 1786204319784630 us; error 25 us; skew 500 ppm
Aug 08 15:51:59 dist-test-slave-sb2z krb5kdc[4256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204319, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:51:59.787497 4408 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:51:59.788262 4408 fs_manager.cc:359] Metadata directory not provided
I20260808 15:51:59.788359 4408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:51:59.788594 4408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:51:59.790761 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "0fc365ebbcb7406b8b175510b3bd5e44"
format_stamp: "Formatted at 2026-08-08 15:51:59 on dist-test-slave-sb2z"
server_key: "91c384db805d3ab2c37781c99c5ba58e"
server_key_iv: "66d6fc5c30c7a3b45e20a383e9b4a9cf"
server_key_version: "encryptionkey@0"
I20260808 15:51:59.791361 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "0fc365ebbcb7406b8b175510b3bd5e44"
format_stamp: "Formatted at 2026-08-08 15:51:59 on dist-test-slave-sb2z"
server_key: "91c384db805d3ab2c37781c99c5ba58e"
server_key_iv: "66d6fc5c30c7a3b45e20a383e9b4a9cf"
server_key_version: "encryptionkey@0"
I20260808 15:51:59.794948 4408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:51:59.797250 4429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:59.798393 4408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:51:59.798525 4408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "0fc365ebbcb7406b8b175510b3bd5e44"
format_stamp: "Formatted at 2026-08-08 15:51:59 on dist-test-slave-sb2z"
server_key: "91c384db805d3ab2c37781c99c5ba58e"
server_key_iv: "66d6fc5c30c7a3b45e20a383e9b4a9cf"
server_key_version: "encryptionkey@0"
I20260808 15:51:59.798631 4408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:51:59.834214 4408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:51:59.837227 4408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:51:59.837431 4408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:51:59.838016 4408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:51:59.838837 4408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:51:59.838907 4408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:59.838980 4408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:51:59.839023 4408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:51:59.848119 4408 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:33187
I20260808 15:51:59.848138 4588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:33187 every 8 connection(s)
I20260808 15:51:59.848706 4408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:51:59.855348 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 4408
I20260808 15:51:59.855460 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:51:59.855801 31025 external_mini_cluster.cc:1428] Setting key bbe9aef1aa771098e95dabe3b6718fa4
Aug 08 15:51:59 dist-test-slave-sb2z krb5kdc[4256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204319, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:51:59.862713 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:59.850405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:44805 (local address 127.30.76.126:45511)
0808 15:51:59.850548 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 15:51:59.850552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:51:59.851396 (+ 844us) server_negotiation.cc:407] Connection header received
0808 15:51:59.852735 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:51:59.852739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:51:59.852799 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:51:59.852904 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:51:59.854060 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:59.854591 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:59.855824 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:51:59.856112 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:51:59.856234 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:59.858768 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:51:59.858772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:51:59.858775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:51:59.858805 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:51:59.860308 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:59.860950 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:59.860953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:59.860954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:59.861029 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:51:59.861492 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:51:59.861495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:51:59.861496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:51:59.861680 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:51:59.861761 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:51:59.862439 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 15:51:59.862584 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:51:59.863942 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:51:59.850797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45511 (local address 127.30.76.65:44805)
0808 15:51:59.851247 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 15:51:59.851288 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:51:59.852513 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:51:59.853074 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:51:59.853082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:51:59.853454 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:51:59.853907 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:59.853920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:59.854720 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:59.854725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:59.855702 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:51:59.855713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:51:59.856276 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:51:59.856280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:51:59.856452 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:51:59.857031 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:51:59.857054 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:51:59.858599 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:51:59.860433 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:59.860440 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:59.860455 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:59.860843 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:59.861124 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:51:59.861130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:51:59.861134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:51:59.861386 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:51:59.861792 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:51:59.861802 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:51:59.862136 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 15:51:59.862384 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 15:51:59.862764 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260808 15:51:59.865233 4589 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45511
I20260808 15:51:59.865471 4589 heartbeater.cc:464] Registering TS with master...
I20260808 15:51:59.865935 4589 heartbeater.cc:511] Master 127.30.76.126:45511 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:51:59.867461 4316 ts_manager.cc:194] Registered new tserver with Master: 0fc365ebbcb7406b8b175510b3bd5e44 (127.30.76.65:33187)
I20260808 15:51:59.868646 4316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:44805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:51:59.884268 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45511
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:51:59.998180 4596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:51:59.998471 4596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:51:59.998561 4596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:00.002825 4596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:00.002908 4596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:00.003036 4596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:00.008407 4596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:00.009464 4596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:00.010484 4596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:00.019742 4602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:00.019793 4601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:00.019742 4604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:00.020073 4596 server_base.cc:1034] running on GCE node
I20260808 15:52:00.020524 4596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:00.021226 4596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:00.022408 4596 hybrid_clock.cc:648] HybridClock initialized: now 1786204320022392 us; error 27 us; skew 500 ppm
Aug 08 15:52:00 dist-test-slave-sb2z krb5kdc[4256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204320, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:00.025202 4596 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:52:00.025931 4596 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:00.026027 4596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:00.026263 4596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:00.028334 4596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "e54411b9e118450c9dec6488fcb5789f"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "065b6795a0e1a4fb05c1a33391642cce"
server_key_iv: "2a8c6794bc3096ad9d69075b8fd41e64"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.028903 4596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "e54411b9e118450c9dec6488fcb5789f"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "065b6795a0e1a4fb05c1a33391642cce"
server_key_iv: "2a8c6794bc3096ad9d69075b8fd41e64"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.032611 4596 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 15:52:00.034948 4612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.036224 4596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:00.036362 4596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "e54411b9e118450c9dec6488fcb5789f"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "065b6795a0e1a4fb05c1a33391642cce"
server_key_iv: "2a8c6794bc3096ad9d69075b8fd41e64"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.036473 4596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:00.054884 4596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:00.057852 4596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:00.058055 4596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:00.058656 4596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:00.059489 4596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:00.059585 4596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.059675 4596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:00.059727 4596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.068300 4596 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:41737
I20260808 15:52:00.068324 4774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:41737 every 8 connection(s)
I20260808 15:52:00.068877 4596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:00.071033 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 4596
I20260808 15:52:00.071190 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:00.071565 31025 external_mini_cluster.cc:1428] Setting key 2c714dbf8acb8ed12feb8919bb4e06e4
Aug 08 15:52:00 dist-test-slave-sb2z krb5kdc[4256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204320, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:52:00.083348 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.070496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:40687 (local address 127.30.76.126:45511)
0808 15:52:00.070666 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.070671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.071687 (+ 1016us) server_negotiation.cc:407] Connection header received
0808 15:52:00.073325 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.073329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.073380 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.073472 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:00.074569 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.075387 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.076539 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.076783 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.076894 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.079757 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:00.079760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:00.079762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:00.079786 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:00.081280 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:00.081806 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:00.081809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:00.081810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:00.081885 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:00.082232 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:00.082235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:00.082237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:00.082418 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:00.082528 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.083052 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.083173 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:52:00.084223 4778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.070942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45511 (local address 127.30.76.66:40687)
0808 15:52:00.071483 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.071570 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.073126 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.073620 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.073627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.073974 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:00.074423 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.074436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.075517 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.075521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.076412 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.076421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.076941 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.076946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.077119 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.077738 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:00.077753 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:00.079580 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:00.081409 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:00.081414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:00.081429 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:00.081689 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:00.081979 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:00.081983 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:00.081985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:00.082130 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:00.082526 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:00.082533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:00.082782 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.082961 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.083289 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":158,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:52:00.085474 4775 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45511
I20260808 15:52:00.085716 4775 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:00.086153 4775 heartbeater.cc:511] Master 127.30.76.126:45511 requested a full tablet report, sending...
I20260808 15:52:00.087342 4316 ts_manager.cc:194] Registered new tserver with Master: e54411b9e118450c9dec6488fcb5789f (127.30.76.66:41737)
I20260808 15:52:00.087957 4316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:40687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:52:00.100356 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45511
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:00.210425 4783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:00.210713 4783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:00.210789 4783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:00.214728 4783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:00.214810 4783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:00.214910 4783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:00.220065 4783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:00.221065 4783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:00.222059 4783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:00.231153 4792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:00.231158 4795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:00.231168 4793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:00.231585 4783 server_base.cc:1034] running on GCE node
I20260808 15:52:00.232023 4783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:00.232637 4783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:00.233827 4783 hybrid_clock.cc:648] HybridClock initialized: now 1786204320233799 us; error 44 us; skew 500 ppm
Aug 08 15:52:00 dist-test-slave-sb2z krb5kdc[4256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204320, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:00.236555 4783 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:52:00.237278 4783 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:00.237373 4783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:00.237599 4783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:00.239745 4783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "e406c186e873422baa9270063d15b5f4"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "1bf6ae25e212b0e7c92d5515415ad9e4"
server_key_iv: "9646e606dba7f53ccaf9bc17d638087c"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.240335 4783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "e406c186e873422baa9270063d15b5f4"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "1bf6ae25e212b0e7c92d5515415ad9e4"
server_key_iv: "9646e606dba7f53ccaf9bc17d638087c"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.244007 4783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:52:00.246320 4805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.247498 4783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:00.247676 4783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "e406c186e873422baa9270063d15b5f4"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "1bf6ae25e212b0e7c92d5515415ad9e4"
server_key_iv: "9646e606dba7f53ccaf9bc17d638087c"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.247784 4783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:00.259763 4783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:00.262441 4783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:00.262631 4783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:00.263185 4783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:00.264098 4783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:00.264176 4783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.264253 4783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:00.264297 4783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.272660 4783 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:37657
I20260808 15:52:00.272707 4971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:37657 every 8 connection(s)
I20260808 15:52:00.273253 4783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:00.277230 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 4783
I20260808 15:52:00.277390 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:00.277701 31025 external_mini_cluster.cc:1428] Setting key 31dc840fc8389acde3077f3f6b70f3ce
Aug 08 15:52:00 dist-test-slave-sb2z krb5kdc[4256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204320, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:00.286430 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.274883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:60035 (local address 127.30.76.126:45511)
0808 15:52:00.275017 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.275020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.275863 (+ 843us) server_negotiation.cc:407] Connection header received
0808 15:52:00.277246 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.277249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.277309 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.277413 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:00.278922 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.279749 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.280832 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.281049 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.281128 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.283289 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:00.283292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:00.283294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:00.283317 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:00.284752 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:00.285188 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:00.285190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:00.285191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:00.285264 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:00.285566 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:00.285568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:00.285569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:00.285737 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:00.285811 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.286230 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.286319 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:00.287341 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.275222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45511 (local address 127.30.76.67:60035)
0808 15:52:00.275701 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.275740 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.277033 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.277611 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.277622 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.278104 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:00.278766 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.278788 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.279874 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.279877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.280723 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.280733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.281179 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.281182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.281311 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.281776 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:00.281791 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:00.283147 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:00.284854 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:00.284858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:00.284869 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:00.285098 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:00.285352 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:00.285354 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:00.285355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:00.285490 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:00.285821 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:00.285826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:00.286027 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.286180 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.286399 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260808 15:52:00.288398 4973 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45511
I20260808 15:52:00.288645 4973 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:00.289054 4973 heartbeater.cc:511] Master 127.30.76.126:45511 requested a full tablet report, sending...
I20260808 15:52:00.290095 4316 ts_manager.cc:194] Registered new tserver with Master: e406c186e873422baa9270063d15b5f4 (127.30.76.67:37657)
I20260808 15:52:00.290866 4316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:60035
I20260808 15:52:00.291617 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:52:00.300642 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.293571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.30.76.126:45511)
0808 15:52:00.293709 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.293715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.293905 (+ 190us) server_negotiation.cc:407] Connection header received
0808 15:52:00.294037 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.294040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.294080 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.294179 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:00.294836 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.295309 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.296205 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.296432 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.296541 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.297788 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:00.297791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:00.297794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:00.297817 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:00.298953 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:00.299376 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:00.299380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:00.299382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:00.299466 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:00.299776 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:00.299780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:00.299782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:00.299986 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:00.300126 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.300368 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.300480 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:52:00.303514 4314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:00.306480 4314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:00.319707 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.314383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:41737 (local address 127.0.0.1:60060)
0808 15:52:00.315428 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.315441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.315556 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.316014 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.316020 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.316041 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:00.316182 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.316189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.317299 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.317304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.318272 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.318282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.319115 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.319118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.319238 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.319282 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.319429 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.319565 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":105,"threads_started":1}
I20260808 15:52:00.319739 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.315022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.30.76.66:41737)
0808 15:52:00.315446 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.315461 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.315509 (+ 48us) server_negotiation.cc:407] Connection header received
0808 15:52:00.315709 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.315714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.315866 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.316022 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:00.316295 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.317181 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.318399 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.318983 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.319101 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.319245 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.319460 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.319580 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":118,"threads_started":1}
I20260808 15:52:00.320658 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.314281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:33187 (local address 127.0.0.1:47256)
0808 15:52:00.315029 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.315077 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.315218 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.316487 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.316490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.316510 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:00.316607 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.316612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.318022 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.318025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.319080 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.319090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.320240 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.320247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.320358 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.320382 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.320470 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.320536 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":58,"threads_started":1}
I20260808 15:52:00.320752 4987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.314399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.30.76.65:33187)
0808 15:52:00.316078 (+ 1679us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.316084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.316104 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:52:00.316176 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.316180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.316347 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.316701 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:00.316872 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.317876 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.319226 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.320133 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.320252 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.320388 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.320495 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.320583 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1579,"thread_start_us":82,"threads_started":1}
I20260808 15:52:00.321484 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.314621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:37657 (local address 127.0.0.1:50102)
0808 15:52:00.315714 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.315725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.315843 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.316294 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.316297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.316313 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:00.316428 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.316439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.318523 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.318526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.319743 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.319754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.321087 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.321092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.321207 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.321231 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.321293 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.321351 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":108,"threads_started":1}
I20260808 15:52:00.321646 4992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.315065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.30.76.67:37657)
0808 15:52:00.315574 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.315590 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.315816 (+ 226us) server_negotiation.cc:407] Connection header received
0808 15:52:00.315951 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.315957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.316143 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.316578 (+ 435us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:00.316717 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.318400 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.319898 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.320937 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.321209 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.321353 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.321400 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.321479 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":137,"threads_started":1}
I20260808 15:52:00.323088 4679 tablet_service.cc:1467] Processing CreateTablet for tablet 0898f2cedd75453998d0642252096273 (DEFAULT_TABLE table=test-table [id=87d6ff632efb4e0a9c16cd79197ca01e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:00.324059 4679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0898f2cedd75453998d0642252096273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:00.323666 4501 tablet_service.cc:1467] Processing CreateTablet for tablet 0898f2cedd75453998d0642252096273 (DEFAULT_TABLE table=test-table [id=87d6ff632efb4e0a9c16cd79197ca01e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:00.324255 4879 tablet_service.cc:1467] Processing CreateTablet for tablet 0898f2cedd75453998d0642252096273 (DEFAULT_TABLE table=test-table [id=87d6ff632efb4e0a9c16cd79197ca01e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:00.324680 4501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0898f2cedd75453998d0642252096273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:00.325217 4879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0898f2cedd75453998d0642252096273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:00.330017 4994 tablet_bootstrap.cc:492] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: Bootstrap starting.
I20260808 15:52:00.331570 4995 tablet_bootstrap.cc:492] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: Bootstrap starting.
I20260808 15:52:00.332368 4996 tablet_bootstrap.cc:492] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: Bootstrap starting.
I20260808 15:52:00.332816 4994 tablet_bootstrap.cc:654] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:00.334199 4994 log.cc:824] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:00.334681 4996 tablet_bootstrap.cc:654] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:00.336041 4996 log.cc:824] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:00.336095 4994 tablet_bootstrap.cc:492] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: No bootstrap required, opened a new log
I20260808 15:52:00.336313 4994 ts_tablet_manager.cc:1397] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260808 15:52:00.336800 4995 tablet_bootstrap.cc:654] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:00.338112 4996 tablet_bootstrap.cc:492] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: No bootstrap required, opened a new log
I20260808 15:52:00.338127 4995 log.cc:824] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:00.338306 4996 ts_tablet_manager.cc:1397] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:52:00.339752 4994 raft_consensus.cc:348] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.339994 4994 raft_consensus.cc:374] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:00.340066 4994 raft_consensus.cc:729] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54411b9e118450c9dec6488fcb5789f, State: Initialized, Role: FOLLOWER
I20260808 15:52:00.340102 4995 tablet_bootstrap.cc:492] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: No bootstrap required, opened a new log
I20260808 15:52:00.340283 4995 ts_tablet_manager.cc:1397] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.002s
I20260808 15:52:00.340528 4994 consensus_queue.cc:260] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.341207 4775 heartbeater.cc:503] Master 127.30.76.126:45511 was elected leader, sending a full tablet report...
I20260808 15:52:00.341534 4994 ts_tablet_manager.cc:1428] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 15:52:00.343410 4996 raft_consensus.cc:348] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.343703 4996 raft_consensus.cc:374] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:00.343763 4996 raft_consensus.cc:729] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fc365ebbcb7406b8b175510b3bd5e44, State: Initialized, Role: FOLLOWER
I20260808 15:52:00.344159 4996 consensus_queue.cc:260] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.344096 4995 raft_consensus.cc:348] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.344340 4995 raft_consensus.cc:374] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:00.344399 4995 raft_consensus.cc:729] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e406c186e873422baa9270063d15b5f4, State: Initialized, Role: FOLLOWER
I20260808 15:52:00.344842 4995 consensus_queue.cc:260] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.344933 4589 heartbeater.cc:503] Master 127.30.76.126:45511 was elected leader, sending a full tablet report...
I20260808 15:52:00.345175 4996 ts_tablet_manager.cc:1428] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 15:52:00.345554 4973 heartbeater.cc:503] Master 127.30.76.126:45511 was elected leader, sending a full tablet report...
I20260808 15:52:00.345724 4995 ts_tablet_manager.cc:1428] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 15:52:00.349030 5004 raft_consensus.cc:482] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:00.349195 5004 raft_consensus.cc:504] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
W20260808 15:52:00.349751 4591 tablet.cc:2369] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:00.350399 5004 leader_election.cc:288] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fc365ebbcb7406b8b175510b3bd5e44 (127.30.76.65:33187), e54411b9e118450c9dec6488fcb5789f (127.30.76.66:41737)
I20260808 15:52:00.354499 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.350553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:33187 (local address 127.30.76.67:48277)
0808 15:52:00.350959 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.350977 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.351073 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.351318 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.351321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.351338 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:00.351467 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.351475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.352434 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.352437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.353261 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.353270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.354055 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.354062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.354170 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.354200 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.354275 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.354364 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339}
I20260808 15:52:00.354506 4987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.350843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:48277 (local address 127.30.76.65:33187)
0808 15:52:00.350969 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.350972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.351016 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:52:00.351173 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.351176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.351229 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.351307 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:00.351608 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.352317 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.353388 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.353838 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.353931 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.353969 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.354311 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.354387 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:52:00.354521 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.350750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:39037 (local address 127.30.76.66:41737)
0808 15:52:00.350853 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.350857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.351410 (+ 553us) server_negotiation.cc:407] Connection header received
0808 15:52:00.351504 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.351510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.351586 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.351680 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:00.351932 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.352461 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.353507 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.354007 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.354119 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.354175 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.354356 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.354423 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 15:52:00.354529 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.350650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:41737 (local address 127.30.76.67:39037)
0808 15:52:00.351319 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.351333 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:00.351425 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:00.351694 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:00.351697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:00.351715 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:00.351823 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.351830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.352572 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.352575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.353376 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:00.353386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.354126 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.354129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:00.354213 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.354231 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 15:52:00.354329 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 15:52:00.354394 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":126,"threads_started":1}
I20260808 15:52:00.355027 4710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0898f2cedd75453998d0642252096273" candidate_uuid: "e406c186e873422baa9270063d15b5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54411b9e118450c9dec6488fcb5789f" is_pre_election: true
I20260808 15:52:00.355108 4529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0898f2cedd75453998d0642252096273" candidate_uuid: "e406c186e873422baa9270063d15b5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" is_pre_election: true
I20260808 15:52:00.355350 4710 raft_consensus.cc:2454] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e406c186e873422baa9270063d15b5f4 in term 0.
I20260808 15:52:00.355458 4529 raft_consensus.cc:2454] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e406c186e873422baa9270063d15b5f4 in term 0.
I20260808 15:52:00.355875 4808 leader_election.cc:302] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0fc365ebbcb7406b8b175510b3bd5e44, e406c186e873422baa9270063d15b5f4, e54411b9e118450c9dec6488fcb5789f; no voters:
I20260808 15:52:00.356106 5004 raft_consensus.cc:2781] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:00.356206 5004 raft_consensus.cc:482] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:00.356256 5004 raft_consensus.cc:3037] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:00.357429 5004 raft_consensus.cc:504] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.357796 5004 leader_election.cc:288] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [CANDIDATE]: Term 1 election: Requested vote from peers 0fc365ebbcb7406b8b175510b3bd5e44 (127.30.76.65:33187), e54411b9e118450c9dec6488fcb5789f (127.30.76.66:41737)
I20260808 15:52:00.358106 4710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0898f2cedd75453998d0642252096273" candidate_uuid: "e406c186e873422baa9270063d15b5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54411b9e118450c9dec6488fcb5789f"
I20260808 15:52:00.358106 4529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0898f2cedd75453998d0642252096273" candidate_uuid: "e406c186e873422baa9270063d15b5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fc365ebbcb7406b8b175510b3bd5e44"
I20260808 15:52:00.358246 4710 raft_consensus.cc:3037] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:00.358259 4529 raft_consensus.cc:3037] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:00.359242 4529 raft_consensus.cc:2454] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e406c186e873422baa9270063d15b5f4 in term 1.
I20260808 15:52:00.359263 4710 raft_consensus.cc:2454] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e406c186e873422baa9270063d15b5f4 in term 1.
I20260808 15:52:00.359584 4810 leader_election.cc:302] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fc365ebbcb7406b8b175510b3bd5e44, e406c186e873422baa9270063d15b5f4; no voters:
I20260808 15:52:00.359788 5004 raft_consensus.cc:2781] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:00.360000 5004 raft_consensus.cc:686] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 1 LEADER]: Becoming Leader. State: Replica: e406c186e873422baa9270063d15b5f4, State: Running, Role: LEADER
I20260808 15:52:00.360373 5004 consensus_queue.cc:237] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } }
I20260808 15:52:00.363005 4316 catalog_manager.cc:5495] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e406c186e873422baa9270063d15b5f4 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "e406c186e873422baa9270063d15b5f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e406c186e873422baa9270063d15b5f4" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37657 } health_report { overall_health: HEALTHY } } }
I20260808 15:52:00.376022 4992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.372352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.30.76.67:37657)
0808 15:52:00.372495 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.372502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.372517 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:00.372702 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.372705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.372766 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.372880 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:00.373138 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.373636 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.374502 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:00.374736 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:00.374877 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:00.375202 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:00.375614 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:00.375744 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.375805 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.375856 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:52:00.382627 4529 raft_consensus.cc:1261] T 0898f2cedd75453998d0642252096273 P 0fc365ebbcb7406b8b175510b3bd5e44 [term 1 FOLLOWER]: Refusing update from remote peer e406c186e873422baa9270063d15b5f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:00.382629 4710 raft_consensus.cc:1261] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 1 FOLLOWER]: Refusing update from remote peer e406c186e873422baa9270063d15b5f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:00.383440 5007 consensus_queue.cc:1035] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54411b9e118450c9dec6488fcb5789f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 41737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:00.383738 5004 consensus_queue.cc:1035] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fc365ebbcb7406b8b175510b3bd5e44" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:00.394145 5013 mvcc.cc:205] Tried to move back new op lower bound from 7316292896279756800 to 7316292896198979584. Current Snapshot: MvccSnapshot[applied={T|T < 7316292896279756800}]
I20260808 15:52:00.394805 5015 mvcc.cc:205] Tried to move back new op lower bound from 7316292896279756800 to 7316292896198979584. Current Snapshot: MvccSnapshot[applied={T|T < 7316292896279756800}]
I20260808 15:52:00.401604 4314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:00.401769 4314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:00.404383 4314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f33fc1b809e8421c98ff02cdc4277478: Sending DeleteTablet for 3 replicas of tablet 0898f2cedd75453998d0642252096273
I20260808 15:52:00.405216 4679 tablet_service.cc:1514] Processing DeleteTablet for tablet 0898f2cedd75453998d0642252096273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:00 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:60060
I20260808 15:52:00.405205 4501 tablet_service.cc:1514] Processing DeleteTablet for tablet 0898f2cedd75453998d0642252096273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:00 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:47256
I20260808 15:52:00.405314 4879 tablet_service.cc:1514] Processing DeleteTablet for tablet 0898f2cedd75453998d0642252096273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:00 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:50102
I20260808 15:52:00.405753 5025 tablet_replica.cc:330] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: stopping tablet replica
I20260808 15:52:00.406077 5025 raft_consensus.cc:2229] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:00.406402 5025 raft_consensus.cc:2258] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:00.406498 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 4408
I20260808 15:52:00.408963 5027 tablet_replica.cc:330] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: stopping tablet replica
I20260808 15:52:00.409350 5027 raft_consensus.cc:2229] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:00.409771 5027 raft_consensus.cc:2258] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:00.413589 5025 ts_tablet_manager.cc:1905] T 0898f2cedd75453998d0642252096273 P e54411b9e118450c9dec6488fcb5789f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:00.415916 5027 ts_tablet_manager.cc:1905] T 0898f2cedd75453998d0642252096273 P e406c186e873422baa9270063d15b5f4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 15:52:00.422060 4296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:33187 (error 108)
I20260808 15:52:00.423026 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 4596
I20260808 15:52:00.423213 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.422907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:33187 (local address 127.0.0.1:47266)
0808 15:52:00.423047 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.423116 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:33187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 15:52:00.430677 4296 catalog_manager.cc:4584] TS 0fc365ebbcb7406b8b175510b3bd5e44 (127.30.76.65:33187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0898f2cedd75453998d0642252096273: Network error: Client connection negotiation failed: client connection to 127.30.76.65:33187: connect: Connection refused (error 111)
I20260808 15:52:00.441356 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 4783
I20260808 15:52:00.441589 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.441267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:41737 (local address 127.0.0.1:60072)
0808 15:52:00.441412 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.441484 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.66:41737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 15:52:00.484663 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.484294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:33187 (local address 127.0.0.1:47280)
0808 15:52:00.484470 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.484540 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:33187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 15:52:00.485431 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.485227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:33187 (local address 127.0.0.1:47292)
0808 15:52:00.485326 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.485373 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:33187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 15:52:00.497102 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.496843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:41737 (local address 127.0.0.1:60088)
0808 15:52:00.496978 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.497033 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.66:41737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 15:52:00.497797 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.497595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:41737 (local address 127.0.0.1:60100)
0808 15:52:00.497701 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:00.497746 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.66:41737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:52:00.525403 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 4277
2026-08-08T15:52:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2581 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T15:52:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:00Z Disabled control of system clock
I20260808 15:52:00.568392 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:33911
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:33911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 15:52:00.685742 5036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:00.686004 5036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:00.686040 5036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:00.690450 5036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:00.690511 5036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:00.690527 5036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:00.690543 5036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:00.690558 5036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:00.695355 5036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:33911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:33911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:00.696408 5036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:00.697758 5036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:00.706302 5043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:00.706447 5036 server_base.cc:1034] running on GCE node
W20260808 15:52:00.706251 5050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:00.706261 5042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:00.707146 5036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:00.708259 5036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:00.709441 5036 hybrid_clock.cc:648] HybridClock initialized: now 1786204320709428 us; error 25 us; skew 500 ppm
I20260808 15:52:00.711246 5036 webserver.cc:460] Webserver started at http://127.30.76.126:41329/ using document root <none> and password file <none>
I20260808 15:52:00.711894 5036 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:00.711992 5036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:00.712232 5036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:00.714325 5036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "73bad8574def465f921b33fe62e8d007"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "42189305608b634c56e63599a784fe9b"
server_key_iv: "c6f1c987d9bbb39f35f50fa25115019b"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.714892 5036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "73bad8574def465f921b33fe62e8d007"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "42189305608b634c56e63599a784fe9b"
server_key_iv: "c6f1c987d9bbb39f35f50fa25115019b"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.718531 5036 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:52:00.720974 5059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:00.722167 5036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:00.722309 5036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "73bad8574def465f921b33fe62e8d007"
format_stamp: "Formatted at 2026-08-08 15:52:00 on dist-test-slave-sb2z"
server_key: "42189305608b634c56e63599a784fe9b"
server_key_iv: "c6f1c987d9bbb39f35f50fa25115019b"
server_key_version: "encryptionkey@0"
I20260808 15:52:00.722424 5036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:00.726969 5036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:00.727589 5036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:00.727788 5036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:00.734964 5036 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:33911
I20260808 15:52:00.734987 5138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:33911 every 8 connection(s)
I20260808 15:52:00.736217 5036 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:00.739322 5139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:00.744129 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 5036
I20260808 15:52:00.744263 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:00.744602 31025 external_mini_cluster.cc:1428] Setting key 6832b92f4aa149667ccc1fb38daed4b1
I20260808 15:52:00.745618 5139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007: Bootstrap starting.
I20260808 15:52:00.748203 5139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:00.749152 5139 log.cc:824] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:00.750494 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:00.746150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.30.76.126:33911)
0808 15:52:00.746810 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:00.746825 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:00.746861 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:52:00.747498 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:00.747515 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:00.747946 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:00.748292 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:00.749117 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:00.749122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:00.749133 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:00.749145 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:00.749199 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:00.749215 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:00.749336 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:00.749439 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 15:52:00.749816 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260808 15:52:00.751354 5139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007: No bootstrap required, opened a new log
I20260808 15:52:00.755113 5139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bad8574def465f921b33fe62e8d007" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33911 } }
I20260808 15:52:00.755343 5139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:00.755378 5139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73bad8574def465f921b33fe62e8d007, State: Initialized, Role: FOLLOWER
I20260808 15:52:00.755810 5139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bad8574def465f921b33fe62e8d007" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33911 } }
I20260808 15:52:00.755934 5139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:00.756022 5139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:00.756153 5139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:00.822489 5139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bad8574def465f921b33fe62e8d007" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33911 } }
I20260808 15:52:00.823020 5139 leader_election.cc:302] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73bad8574def465f921b33fe62e8d007; no voters:
I20260808 15:52:00.823411 5139 leader_election.cc:288] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:00.823597 5146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:00.823901 5146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [term 1 LEADER]: Becoming Leader. State: Replica: 73bad8574def465f921b33fe62e8d007, State: Running, Role: LEADER
I20260808 15:52:00.824303 5146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bad8574def465f921b33fe62e8d007" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33911 } }
I20260808 15:52:00.824993 5139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:00.825840 5152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73bad8574def465f921b33fe62e8d007. Latest consensus state: current_term: 1 leader_uuid: "73bad8574def465f921b33fe62e8d007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bad8574def465f921b33fe62e8d007" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33911 } } }
I20260808 15:52:00.826013 5152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:00.826498 5147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73bad8574def465f921b33fe62e8d007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bad8574def465f921b33fe62e8d007" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33911 } } }
I20260808 15:52:00.826615 5147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:00.827396 5157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:00.831326 5157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:00.836788 5157 catalog_manager.cc:1347] Generated new cluster ID: c68e9a56e2174d9c83b2be072e9acf2c
I20260808 15:52:00.836866 5157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:00.852710 5157 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:00.853032 5157 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:00.857699 5157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007: Generated new TSK 0
I20260808 15:52:00.858373 5157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:52:00.864144 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:33911
--builtin_ntp_servers=127.30.76.84:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 15:52:00.984576 5177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:00.984859 5177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:00.984939 5177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:00.989102 5177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:00.989176 5177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:00.989264 5177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:00.995390 5177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:33911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:00.996488 5177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:00.997880 5177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:01.007628 5185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.007665 5188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:01.007681 5177 server_base.cc:1034] running on GCE node
W20260808 15:52:01.007627 5190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:01.008275 5177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:01.008903 5177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:01.010062 5177 hybrid_clock.cc:648] HybridClock initialized: now 1786204321010034 us; error 41 us; skew 500 ppm
I20260808 15:52:01.012142 5177 webserver.cc:460] Webserver started at http://127.30.76.65:32927/ using document root <none> and password file <none>
I20260808 15:52:01.012693 5177 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:01.012766 5177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:01.012940 5177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:01.014988 5177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "0219ff30e03245849594f07f8ea1f3bd"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "4df3f29b4ed2f34bf3adaa910898182b"
server_key_iv: "af5db840b7d2da5e021255fa264c95bd"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.015506 5177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "0219ff30e03245849594f07f8ea1f3bd"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "4df3f29b4ed2f34bf3adaa910898182b"
server_key_iv: "af5db840b7d2da5e021255fa264c95bd"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.019177 5177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:01.021803 5204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.023104 5177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:01.023242 5177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "0219ff30e03245849594f07f8ea1f3bd"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "4df3f29b4ed2f34bf3adaa910898182b"
server_key_iv: "af5db840b7d2da5e021255fa264c95bd"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.023355 5177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:01.028571 5177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:01.029136 5177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:01.029325 5177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:01.029867 5177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:01.030766 5177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:01.030813 5177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.030876 5177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:01.030906 5177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.040670 5177 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:38645
I20260808 15:52:01.040699 5370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:38645 every 8 connection(s)
I20260808 15:52:01.041597 5177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:01.041962 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 5177
I20260808 15:52:01.042099 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:01.042448 31025 external_mini_cluster.cc:1428] Setting key 67d9d8b164f8d961d98780bb22b23201
I20260808 15:52:01.044721 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:33911
--builtin_ntp_servers=127.30.76.84:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 15:52:01.048847 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.043757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:54357 (local address 127.30.76.126:33911)
0808 15:52:01.043907 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.043912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.045212 (+ 1300us) server_negotiation.cc:407] Connection header received
0808 15:52:01.046818 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.046820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.046885 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.046996 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.048308 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.048314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.048320 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.048337 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.048362 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.048375 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.048476 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.048599 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.048660 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:52:01.049618 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.044322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:33911 (local address 127.30.76.65:54357)
0808 15:52:01.045051 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.045102 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.046536 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.047208 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.047220 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.047339 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.047989 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.047993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.048054 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.048148 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.048466 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.048473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.048477 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.048607 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.048802 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":139,"threads_started":1}
I20260808 15:52:01.050869 5371 heartbeater.cc:347] Connected to a master server at 127.30.76.126:33911
I20260808 15:52:01.051132 5371 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:01.051666 5371 heartbeater.cc:511] Master 127.30.76.126:33911 requested a full tablet report, sending...
I20260808 15:52:01.053280 5083 ts_manager.cc:194] Registered new tserver with Master: 0219ff30e03245849594f07f8ea1f3bd (127.30.76.65:38645)
W20260808 15:52:01.167168 5375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:01.167435 5375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:01.167475 5375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:01.171669 5375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:01.171747 5375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:01.171868 5375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:01.177286 5375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:33911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:01.178354 5375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:01.179853 5375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:01.189838 5388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.189851 5389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.189838 5391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:01.190048 5375 server_base.cc:1034] running on GCE node
I20260808 15:52:01.190582 5375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:01.191210 5375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:01.192399 5375 hybrid_clock.cc:648] HybridClock initialized: now 1786204321192363 us; error 69 us; skew 500 ppm
I20260808 15:52:01.194240 5375 webserver.cc:460] Webserver started at http://127.30.76.66:38087/ using document root <none> and password file <none>
I20260808 15:52:01.194828 5375 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:01.194922 5375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:01.195156 5375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:01.197218 5375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "33293ba1cbed48dab0ad03de1bc7abb2"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "e68e3eeb4c4f51475e99dc80185e9fda"
server_key_iv: "a973599bfbcf406712ae0addc697da28"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.197786 5375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "33293ba1cbed48dab0ad03de1bc7abb2"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "e68e3eeb4c4f51475e99dc80185e9fda"
server_key_iv: "a973599bfbcf406712ae0addc697da28"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.201452 5375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 15:52:01.204054 5399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.205305 5375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:01.205442 5375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "33293ba1cbed48dab0ad03de1bc7abb2"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "e68e3eeb4c4f51475e99dc80185e9fda"
server_key_iv: "a973599bfbcf406712ae0addc697da28"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.205560 5375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:01.210121 5375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:01.210743 5375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:01.210927 5375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:01.211465 5375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:01.212464 5375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:01.212532 5375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.212597 5375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:01.212656 5375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.222054 5375 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:34953
I20260808 15:52:01.222115 5560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:34953 every 8 connection(s)
I20260808 15:52:01.223038 5375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:01.229584 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.224920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:46113 (local address 127.30.76.126:33911)
0808 15:52:01.225062 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.225066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.226243 (+ 1177us) server_negotiation.cc:407] Connection header received
0808 15:52:01.227715 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.227718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.227759 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.227903 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.229144 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.229147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.229149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.229161 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.229180 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.229188 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.229267 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.229419 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.229471 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:52:01.230463 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.225278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:33911 (local address 127.30.76.66:46113)
0808 15:52:01.226097 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.226159 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.227461 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.228138 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.228145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.228230 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.228874 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.228876 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.228915 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.228983 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.229269 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.229277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.229281 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.229414 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.229607 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":140,"threads_started":1}
I20260808 15:52:01.231673 5562 heartbeater.cc:347] Connected to a master server at 127.30.76.126:33911
I20260808 15:52:01.231922 5562 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:01.232347 5562 heartbeater.cc:511] Master 127.30.76.126:33911 requested a full tablet report, sending...
I20260808 15:52:01.232831 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 5375
I20260808 15:52:01.232962 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:01.233325 31025 external_mini_cluster.cc:1428] Setting key cca414c166657b6d74b3f6aa3274b5f0
I20260808 15:52:01.233651 5083 ts_manager.cc:194] Registered new tserver with Master: 33293ba1cbed48dab0ad03de1bc7abb2 (127.30.76.66:34953)
I20260808 15:52:01.235880 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:33911
--builtin_ntp_servers=127.30.76.84:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 15:52:01.362313 5566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:01.362603 5566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:01.362685 5566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:01.366901 5566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:01.366998 5566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:01.367113 5566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:01.372134 5566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:33911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:01.373220 5566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:01.374644 5566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:52:01.384824 5566 server_base.cc:1034] running on GCE node
W20260808 15:52:01.384847 5572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.384766 5571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.384788 5574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:01.385413 5566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:01.386068 5566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:01.387257 5566 hybrid_clock.cc:648] HybridClock initialized: now 1786204321387244 us; error 26 us; skew 500 ppm
I20260808 15:52:01.389138 5566 webserver.cc:460] Webserver started at http://127.30.76.67:35291/ using document root <none> and password file <none>
I20260808 15:52:01.389880 5566 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:01.389977 5566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:01.390195 5566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:01.392390 5566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "9e1142aa65314d01b1588938f76087d1"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "d1b78b3233b23f4e2d37a440d5580d41"
server_key_iv: "72334f17d49493fbb322524db4e7ba29"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.392969 5566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "9e1142aa65314d01b1588938f76087d1"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "d1b78b3233b23f4e2d37a440d5580d41"
server_key_iv: "72334f17d49493fbb322524db4e7ba29"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.397110 5566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:52:01.399988 5582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.401248 5566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:01.401386 5566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "9e1142aa65314d01b1588938f76087d1"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "d1b78b3233b23f4e2d37a440d5580d41"
server_key_iv: "72334f17d49493fbb322524db4e7ba29"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.401501 5566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:01.406237 5566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:01.406842 5566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:01.407032 5566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:01.407626 5566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:01.408591 5566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:01.408669 5566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.408766 5566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:01.408819 5566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.417833 5566 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:36843
I20260808 15:52:01.417871 5753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:36843 every 8 connection(s)
I20260808 15:52:01.418728 5566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:01.422947 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 5566
I20260808 15:52:01.423077 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:01.423425 31025 external_mini_cluster.cc:1428] Setting key fb9da11819981564071d8e6aff72276b
I20260808 15:52:01.425053 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.420566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:46415 (local address 127.30.76.126:33911)
0808 15:52:01.420727 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.420731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.421826 (+ 1095us) server_negotiation.cc:407] Connection header received
0808 15:52:01.423148 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.423151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.423201 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.423302 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.424545 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.424549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.424552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.424565 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.424588 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.424601 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.424705 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.424892 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.424932 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 15:52:01.426054 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.420985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:33911 (local address 127.30.76.67:46415)
0808 15:52:01.421683 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.421722 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.422931 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.423483 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.423490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.423640 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.424238 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.424239 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.424278 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.424394 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.424746 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.424758 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.424768 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.424923 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.425105 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260808 15:52:01.427176 5755 heartbeater.cc:347] Connected to a master server at 127.30.76.126:33911
I20260808 15:52:01.427414 5755 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:01.427845 5755 heartbeater.cc:511] Master 127.30.76.126:33911 requested a full tablet report, sending...
I20260808 15:52:01.428845 5081 ts_manager.cc:194] Registered new tserver with Master: 9e1142aa65314d01b1588938f76087d1 (127.30.76.67:36843)
I20260808 15:52:01.437263 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:52:01.440765 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.439193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.30.76.126:33911)
0808 15:52:01.439321 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.439333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.439518 (+ 185us) server_negotiation.cc:407] Connection header received
0808 15:52:01.439688 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.439690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.439723 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.439790 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.440490 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.440493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.440494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.440502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.440516 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.440526 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.440586 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.440629 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.440669 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:52:01.443305 5081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:01.446434 5081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:01.458235 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.454941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:38645 (local address 127.0.0.1:45714)
0808 15:52:01.455476 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.455551 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.455649 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.456297 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.456305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.456381 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.456975 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.456978 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.456998 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.457064 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.457881 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.457891 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.457897 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.457988 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.458058 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":74,"threads_started":1}
I20260808 15:52:01.458568 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.455120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36843 (local address 127.0.0.1:40582)
0808 15:52:01.456146 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.456160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.456296 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.457275 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.457277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.457293 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.457734 (+ 441us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.457737 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.457749 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.457784 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.458125 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.458128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.458132 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.458201 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.458237 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"thread_start_us":119,"threads_started":1}
I20260808 15:52:01.458580 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.455662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.30.76.67:36843)
0808 15:52:01.456000 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.456012 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.456237 (+ 225us) server_negotiation.cc:407] Connection header received
0808 15:52:01.456420 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.456429 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.456637 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.456748 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.457937 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.457943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.457953 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.457965 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.458011 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.458033 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.458162 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.458211 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.458282 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
I20260808 15:52:01.458812 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.455830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:34953 (local address 127.0.0.1:39944)
0808 15:52:01.456418 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.456429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.456486 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.457360 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.457365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.457382 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.458087 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.458089 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.458097 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.458120 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.458552 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.458556 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.458560 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.458643 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.458686 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":99,"threads_started":1}
I20260808 15:52:01.458952 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.455095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.30.76.65:38645)
0808 15:52:01.455801 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.455813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.455841 (+ 28us) server_negotiation.cc:407] Connection header received
0808 15:52:01.455896 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.455901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.456088 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.456690 (+ 602us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.457660 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.457666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.457678 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.457688 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.457729 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.457761 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.458545 (+ 784us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.458592 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.458791 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":143,"threads_started":1}
I20260808 15:52:01.459208 5778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.456131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.30.76.66:34953)
0808 15:52:01.456809 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.456821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.456845 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:52:01.456895 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.456900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.457056 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.457202 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.458248 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.458256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.458269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.458282 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.458321 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.458397 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.458507 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.458974 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.459042 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":146,"threads_started":1}
I20260808 15:52:01.461364 5655 tablet_service.cc:1467] Processing CreateTablet for tablet fb38a45bdd294fceadbf7c6c2b514bbd (DEFAULT_TABLE table=test-table [id=49dcb7f6a4cb4da58e23dfbd6e286292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:01.461855 5271 tablet_service.cc:1467] Processing CreateTablet for tablet fb38a45bdd294fceadbf7c6c2b514bbd (DEFAULT_TABLE table=test-table [id=49dcb7f6a4cb4da58e23dfbd6e286292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:01.461858 5464 tablet_service.cc:1467] Processing CreateTablet for tablet fb38a45bdd294fceadbf7c6c2b514bbd (DEFAULT_TABLE table=test-table [id=49dcb7f6a4cb4da58e23dfbd6e286292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:01.462368 5655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb38a45bdd294fceadbf7c6c2b514bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:01.462826 5464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb38a45bdd294fceadbf7c6c2b514bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:01.462826 5271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb38a45bdd294fceadbf7c6c2b514bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:01.468690 5779 tablet_bootstrap.cc:492] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Bootstrap starting.
I20260808 15:52:01.470229 5780 tablet_bootstrap.cc:492] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: Bootstrap starting.
I20260808 15:52:01.470656 5781 tablet_bootstrap.cc:492] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: Bootstrap starting.
I20260808 15:52:01.472021 5779 tablet_bootstrap.cc:654] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:01.472424 5780 tablet_bootstrap.cc:654] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:01.472779 5781 tablet_bootstrap.cc:654] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:01.472963 5779 log.cc:824] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:01.473224 5780 log.cc:824] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:01.473543 5781 log.cc:824] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:01.475099 5780 tablet_bootstrap.cc:492] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: No bootstrap required, opened a new log
I20260808 15:52:01.475123 5779 tablet_bootstrap.cc:492] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: No bootstrap required, opened a new log
I20260808 15:52:01.475250 5780 ts_tablet_manager.cc:1397] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:52:01.475301 5779 ts_tablet_manager.cc:1397] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:52:01.475473 5781 tablet_bootstrap.cc:492] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: No bootstrap required, opened a new log
I20260808 15:52:01.475667 5781 ts_tablet_manager.cc:1397] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 15:52:01.479211 5780 raft_consensus.cc:348] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.479440 5780 raft_consensus.cc:374] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:01.479492 5780 raft_consensus.cc:729] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0219ff30e03245849594f07f8ea1f3bd, State: Initialized, Role: FOLLOWER
I20260808 15:52:01.479940 5780 consensus_queue.cc:260] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.479828 5779 raft_consensus.cc:348] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.480077 5779 raft_consensus.cc:374] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:01.480144 5779 raft_consensus.cc:729] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1142aa65314d01b1588938f76087d1, State: Initialized, Role: FOLLOWER
I20260808 15:52:01.480017 5781 raft_consensus.cc:348] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.480261 5781 raft_consensus.cc:374] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:01.480307 5781 raft_consensus.cc:729] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33293ba1cbed48dab0ad03de1bc7abb2, State: Initialized, Role: FOLLOWER
I20260808 15:52:01.480597 5779 consensus_queue.cc:260] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.480652 5781 consensus_queue.cc:260] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.481002 5371 heartbeater.cc:503] Master 127.30.76.126:33911 was elected leader, sending a full tablet report...
I20260808 15:52:01.481272 5780 ts_tablet_manager.cc:1428] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:52:01.481575 5779 ts_tablet_manager.cc:1428] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:52:01.481575 5781 ts_tablet_manager.cc:1428] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 15:52:01.481914 5562 heartbeater.cc:503] Master 127.30.76.126:33911 was elected leader, sending a full tablet report...
I20260808 15:52:01.482192 5755 heartbeater.cc:503] Master 127.30.76.126:33911 was elected leader, sending a full tablet report...
I20260808 15:52:01.532691 5792 raft_consensus.cc:482] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:01.532924 5792 raft_consensus.cc:504] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.534046 5792 leader_election.cc:288] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0219ff30e03245849594f07f8ea1f3bd (127.30.76.65:38645), 33293ba1cbed48dab0ad03de1bc7abb2 (127.30.76.66:34953)
I20260808 15:52:01.536083 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.534325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:38645 (local address 127.30.76.67:58105)
0808 15:52:01.534460 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.534477 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.534560 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.534838 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.534842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.534858 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.535438 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.535440 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.535453 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.535482 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.535812 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.535816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.535822 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.535918 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.535959 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 15:52:01.536121 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.534384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:58105 (local address 127.30.76.65:38645)
0808 15:52:01.534525 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.534529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.534545 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:01.534658 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.534662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.534705 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.534820 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.535656 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.535660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.535664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.535679 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.535706 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.535720 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.535817 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.535911 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.535976 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:52:01.536459 5778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.534771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:57141 (local address 127.30.76.66:34953)
0808 15:52:01.534901 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.534907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.535145 (+ 238us) server_negotiation.cc:407] Connection header received
0808 15:52:01.535215 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.535219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.535264 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.535345 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.536007 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.536010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.536013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.536025 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.536058 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.536068 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.536170 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.536319 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.536365 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:52:01.536480 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.534610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:34953 (local address 127.30.76.67:57141)
0808 15:52:01.535045 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.535057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:01.535140 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:01.535393 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:01.535397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:01.535418 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:01.535838 (+ 420us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:01.535840 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:01.535852 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:01.535882 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:01.536180 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:01.536184 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:01.536187 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 15:52:01.536314 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 15:52:01.536355 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":163,"threads_started":1}
I20260808 15:52:01.537068 5495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb38a45bdd294fceadbf7c6c2b514bbd" candidate_uuid: "9e1142aa65314d01b1588938f76087d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" is_pre_election: true
I20260808 15:52:01.537392 5495 raft_consensus.cc:2454] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1142aa65314d01b1588938f76087d1 in term 0.
I20260808 15:52:01.537874 5303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb38a45bdd294fceadbf7c6c2b514bbd" candidate_uuid: "9e1142aa65314d01b1588938f76087d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0219ff30e03245849594f07f8ea1f3bd" is_pre_election: true
I20260808 15:52:01.537921 5583 leader_election.cc:302] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33293ba1cbed48dab0ad03de1bc7abb2, 9e1142aa65314d01b1588938f76087d1; no voters:
I20260808 15:52:01.538172 5303 raft_consensus.cc:2454] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1142aa65314d01b1588938f76087d1 in term 0.
I20260808 15:52:01.538215 5792 raft_consensus.cc:2781] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:01.538301 5792 raft_consensus.cc:482] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:01.538367 5792 raft_consensus.cc:3037] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 0 FOLLOWER]: Advancing to term 1
W20260808 15:52:01.542917 5372 tablet.cc:2369] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:01.603055 5792 raft_consensus.cc:504] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.603704 5792 leader_election.cc:288] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [CANDIDATE]: Term 1 election: Requested vote from peers 0219ff30e03245849594f07f8ea1f3bd (127.30.76.65:38645), 33293ba1cbed48dab0ad03de1bc7abb2 (127.30.76.66:34953)
I20260808 15:52:01.604245 5303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb38a45bdd294fceadbf7c6c2b514bbd" candidate_uuid: "9e1142aa65314d01b1588938f76087d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0219ff30e03245849594f07f8ea1f3bd"
I20260808 15:52:01.604291 5495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb38a45bdd294fceadbf7c6c2b514bbd" candidate_uuid: "9e1142aa65314d01b1588938f76087d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33293ba1cbed48dab0ad03de1bc7abb2"
I20260808 15:52:01.604418 5495 raft_consensus.cc:3037] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:01.604418 5303 raft_consensus.cc:3037] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:01.605923 5303 raft_consensus.cc:2454] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1142aa65314d01b1588938f76087d1 in term 1.
I20260808 15:52:01.605926 5495 raft_consensus.cc:2454] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1142aa65314d01b1588938f76087d1 in term 1.
I20260808 15:52:01.606374 5583 leader_election.cc:302] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33293ba1cbed48dab0ad03de1bc7abb2, 9e1142aa65314d01b1588938f76087d1; no voters:
I20260808 15:52:01.606695 5792 raft_consensus.cc:2781] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:01.606973 5792 raft_consensus.cc:686] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 1 LEADER]: Becoming Leader. State: Replica: 9e1142aa65314d01b1588938f76087d1, State: Running, Role: LEADER
I20260808 15:52:01.607390 5792 consensus_queue.cc:237] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } }
I20260808 15:52:01.610924 5083 catalog_manager.cc:5495] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e1142aa65314d01b1588938f76087d1 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "9e1142aa65314d01b1588938f76087d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e1142aa65314d01b1588938f76087d1" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 } health_report { overall_health: UNKNOWN } } }
I20260808 15:52:01.649766 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.648179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.30.76.67:36843)
0808 15:52:01.648332 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:01.648337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:01.648352 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:01.648517 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:01.648519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:01.648581 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:01.648690 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:01.649340 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:01.649344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:01.649346 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:01.649358 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:01.649385 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:01.649393 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:01.649487 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:01.649568 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 15:52:01.649607 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:52:01.656548 5495 raft_consensus.cc:1261] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 1 FOLLOWER]: Refusing update from remote peer 9e1142aa65314d01b1588938f76087d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:01.656562 5303 raft_consensus.cc:1261] T fb38a45bdd294fceadbf7c6c2b514bbd P 0219ff30e03245849594f07f8ea1f3bd [term 1 FOLLOWER]: Refusing update from remote peer 9e1142aa65314d01b1588938f76087d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:01.657317 5794 consensus_queue.cc:1035] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33293ba1cbed48dab0ad03de1bc7abb2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:01.657498 5792 consensus_queue.cc:1035] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0219ff30e03245849594f07f8ea1f3bd" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:01.665364 5798 mvcc.cc:205] Tried to move back new op lower bound from 7316292901497384960 to 7316292901306916864. Current Snapshot: MvccSnapshot[applied={T|T < 7316292901497384960}]
I20260808 15:52:01.673370 5081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:01.673566 5081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:01.676040 5081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73bad8574def465f921b33fe62e8d007: Sending DeleteTablet for 3 replicas of tablet fb38a45bdd294fceadbf7c6c2b514bbd
I20260808 15:52:01.676873 5464 tablet_service.cc:1514] Processing DeleteTablet for tablet fb38a45bdd294fceadbf7c6c2b514bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:01 UTC) from {username='slave'} at 127.0.0.1:39944
I20260808 15:52:01.676854 5655 tablet_service.cc:1514] Processing DeleteTablet for tablet fb38a45bdd294fceadbf7c6c2b514bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:01 UTC) from {username='slave'} at 127.0.0.1:40582
I20260808 15:52:01.676980 5271 tablet_service.cc:1514] Processing DeleteTablet for tablet fb38a45bdd294fceadbf7c6c2b514bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:01 UTC) from {username='slave'} at 127.0.0.1:45714
I20260808 15:52:01.677493 5811 tablet_replica.cc:330] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: stopping tablet replica
I20260808 15:52:01.677673 5812 tablet_replica.cc:330] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: stopping tablet replica
I20260808 15:52:01.677933 5811 raft_consensus.cc:2229] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:01.677933 5812 raft_consensus.cc:2229] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:01.678076 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 5177
I20260808 15:52:01.678262 5812 raft_consensus.cc:2258] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:01.679049 5811 raft_consensus.cc:2258] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:01.682021 5812 ts_tablet_manager.cc:1905] T fb38a45bdd294fceadbf7c6c2b514bbd P 33293ba1cbed48dab0ad03de1bc7abb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:01.682021 5811 ts_tablet_manager.cc:1905] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 15:52:01.694520 5061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:38645 (error 108)
I20260808 15:52:01.694751 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 5375
I20260808 15:52:01.695943 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.695584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:38645 (local address 127.0.0.1:45722)
0808 15:52:01.695778 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.695850 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260808 15:52:01.696113 5061 catalog_manager.cc:4584] TS 0219ff30e03245849594f07f8ea1f3bd (127.30.76.65:38645): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb38a45bdd294fceadbf7c6c2b514bbd: Network error: Client connection negotiation failed: client connection to 127.30.76.65:38645: connect: Connection refused (error 111)
I20260808 15:52:01.706156 5811 ts_tablet_manager.cc:1918] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:01.706354 5811 log.cc:1196] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/fb38a45bdd294fceadbf7c6c2b514bbd
I20260808 15:52:01.706727 5811 ts_tablet_manager.cc:1939] T fb38a45bdd294fceadbf7c6c2b514bbd P 9e1142aa65314d01b1588938f76087d1: Deleting consensus metadata
I20260808 15:52:01.709702 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 5566
I20260808 15:52:01.712531 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:01.712130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:34953 (local address 127.0.0.1:39948)
0808 15:52:01.712365 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:01.712424 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.66:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260808 15:52:01.729120 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 5036
2026-08-08T15:52:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1201 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 15:52:01.760064 31025 mini_oidc.cc:152] Starting JWKS server
I20260808 15:52:01.762441 31025 webserver.cc:460] Webserver started at https://127.0.0.1:35297/ using document root <none> and password file <none>
I20260808 15:52:01.762672 31025 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 15:52:01.762995 31025 webserver.cc:460] Webserver started at http://127.0.0.1:35107/ using document root <none> and password file <none>
2026-08-08T15:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:01Z Disabled control of system clock
I20260808 15:52:01.774596 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36785
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:36785
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35297/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/testchainca.cert with env {}
W20260808 15:52:01.893474 5823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:01.893741 5823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:01.893780 5823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:01.897907 5823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:01.897967 5823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:01.897984 5823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:01.898000 5823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:01.898016 5823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:52:01.898032 5823 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:52:01.898046 5823 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35297/jwks.json
I20260808 15:52:01.903740 5823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:36785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35297/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:01.904830 5823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:01.905874 5823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:01.914407 5829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.914399 5835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:01.914399 5833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:01.914947 5823 server_base.cc:1034] running on GCE node
I20260808 15:52:01.915575 5823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:01.916714 5823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:01.917917 5823 hybrid_clock.cc:648] HybridClock initialized: now 1786204321917887 us; error 41 us; skew 500 ppm
I20260808 15:52:01.919852 5823 webserver.cc:460] Webserver started at http://127.30.76.126:46309/ using document root <none> and password file <none>
I20260808 15:52:01.920507 5823 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:01.920571 5823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:01.920801 5823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:01.923074 5823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "c1e7ea33a9834e1f94f1786da674089a"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "6e9de711ee3119a3f8a38b457390329a"
server_key_iv: "97cdbbc0262519ab4fdab36f187f5c9a"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.923729 5823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "c1e7ea33a9834e1f94f1786da674089a"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "6e9de711ee3119a3f8a38b457390329a"
server_key_iv: "97cdbbc0262519ab4fdab36f187f5c9a"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.927407 5823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:01.929792 5846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:01.930950 5823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:01.931088 5823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "c1e7ea33a9834e1f94f1786da674089a"
format_stamp: "Formatted at 2026-08-08 15:52:01 on dist-test-slave-sb2z"
server_key: "6e9de711ee3119a3f8a38b457390329a"
server_key_iv: "97cdbbc0262519ab4fdab36f187f5c9a"
server_key_version: "encryptionkey@0"
I20260808 15:52:01.931207 5823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:01.988921 5823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:01.989768 5823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:01.989985 5823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:01.998497 5823 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:36785
I20260808 15:52:01.998520 5914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:36785 every 8 connection(s)
I20260808 15:52:01.999644 5823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:02.001500 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 5823
I20260808 15:52:02.001652 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:02.001995 31025 external_mini_cluster.cc:1428] Setting key 44b7cd3bc41b3389d289a16f59ba18b0
I20260808 15:52:02.002915 5917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:02.008931 5917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a: Bootstrap starting.
I20260808 15:52:02.011502 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:02.003485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.30.76.126:36785)
0808 15:52:02.004251 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:02.004268 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:02.004303 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:52:02.004981 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:02.004998 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:02.005372 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:02.005726 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:02.006156 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.007134 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.008195 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.008678 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.008856 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.009744 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:02.009750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:02.009762 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:02.010163 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:02.010219 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:02.010240 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:02.010354 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:02.010471 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 15:52:02.010758 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":132,"threads_started":1}
I20260808 15:52:02.012223 5917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:02.013221 5917 log.cc:824] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:02.014849 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:52:02.016656 5917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a: No bootstrap required, opened a new log
I20260808 15:52:02.020468 5917 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e7ea33a9834e1f94f1786da674089a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36785 } }
I20260808 15:52:02.020727 5917 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:02.020823 5917 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e7ea33a9834e1f94f1786da674089a, State: Initialized, Role: FOLLOWER
I20260808 15:52:02.021339 5917 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e7ea33a9834e1f94f1786da674089a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36785 } }
I20260808 15:52:02.021520 5917 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:02.021584 5917 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:02.021680 5917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:02.115377 5917 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e7ea33a9834e1f94f1786da674089a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36785 } }
I20260808 15:52:02.115921 5917 leader_election.cc:302] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1e7ea33a9834e1f94f1786da674089a; no voters:
I20260808 15:52:02.116303 5917 leader_election.cc:288] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:02.116454 5924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:02.116703 5924 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [term 1 LEADER]: Becoming Leader. State: Replica: c1e7ea33a9834e1f94f1786da674089a, State: Running, Role: LEADER
I20260808 15:52:02.117137 5924 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e7ea33a9834e1f94f1786da674089a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36785 } }
I20260808 15:52:02.117712 5917 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:02.119045 5928 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e7ea33a9834e1f94f1786da674089a. Latest consensus state: current_term: 1 leader_uuid: "c1e7ea33a9834e1f94f1786da674089a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e7ea33a9834e1f94f1786da674089a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36785 } } }
I20260808 15:52:02.119215 5928 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:02.119680 5938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:02.119638 5925 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1e7ea33a9834e1f94f1786da674089a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e7ea33a9834e1f94f1786da674089a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36785 } } }
I20260808 15:52:02.119756 5925 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:02.123843 5938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:02.129426 5938 catalog_manager.cc:1347] Generated new cluster ID: ed1e71dc43e444bcb95d57220b37f0e8
I20260808 15:52:02.129501 5938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:02.147354 5938 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:02.148262 5938 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:02.158870 5938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e7ea33a9834e1f94f1786da674089a: Generated new TSK 0
I20260808 15:52:02.159597 5938 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 15:52:02.196861 5960 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 15:52:02.197016 5960 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 15:52:02.197611 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:02.178197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.30.76.126:36785)
0808 15:52:02.178332 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:02.178338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:02.178508 (+ 170us) server_negotiation.cc:407] Connection header received
0808 15:52:02.178710 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:02.178713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:02.178769 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:02.178879 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:52:02.179092 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.179949 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.180760 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.181107 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.181219 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.181511 (+ 292us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:52:02.197218 (+ 15707us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 15:52:02.197475 (+ 257us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 15:52:02.197680 5922 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 15:52:02.198909 5959 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:02.178078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36785 (local address 127.0.0.1:47014)
0808 15:52:02.178442 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:02.178455 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:02.178561 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:02.178894 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:02.178896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:02.178903 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:52:02.178992 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:02.178997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.180056 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.180058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:02.180657 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:02.180665 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.181228 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.181231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:02.181292 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.181332 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:52:02.197422 (+ 16090us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 15:52:02.197606 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.76.126:36785: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.76.126:36785: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 15:52:02.199617 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 5823
2026-08-08T15:52:02Z chronyd exiting
I20260808 15:52:02.245183 31025 test_util.cc:175] -----------------------------------------------
I20260808 15:52:02.245288 31025 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786204294980517-31025-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (487 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 15:52:02.246387 31025 mini_oidc.cc:152] Starting JWKS server
I20260808 15:52:02.247241 31025 webserver.cc:460] Webserver started at https://127.0.0.1:46507/ using document root <none> and password file <none>
I20260808 15:52:02.247370 31025 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 15:52:02.247717 31025 webserver.cc:460] Webserver started at http://127.0.0.1:40269/ using document root <none> and password file <none>
2026-08-08T15:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:02Z Disabled control of system clock
I20260808 15:52:02.257745 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:37013
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:42593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:37013
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46507/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/testchainca.cert with env {}
W20260808 15:52:02.367105 5971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:02.367348 5971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:02.367385 5971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:02.371387 5971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:02.371440 5971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:02.371456 5971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:02.371474 5971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:02.371488 5971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:52:02.371505 5971 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:52:02.371518 5971 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46507/jwks.json
I20260808 15:52:02.376819 5971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:37013
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:37013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46507/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:02.377703 5971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:02.378597 5971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:02.386701 5980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:02.386711 5979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:02.386708 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:02.387089 5971 server_base.cc:1034] running on GCE node
I20260808 15:52:02.387708 5971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:02.388703 5971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:02.389860 5971 hybrid_clock.cc:648] HybridClock initialized: now 1786204322389844 us; error 33 us; skew 500 ppm
I20260808 15:52:02.391853 5971 webserver.cc:460] Webserver started at http://127.30.76.126:40733/ using document root <none> and password file <none>
I20260808 15:52:02.392411 5971 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:02.392473 5971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:02.392729 5971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:02.394652 5971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "46974c6e24c248f4ae385d6a28257046"
format_stamp: "Formatted at 2026-08-08 15:52:02 on dist-test-slave-sb2z"
server_key: "bd9a7a4954226e12b3b609c6b408c2c6"
server_key_iv: "3051e7d524290afdf0520c5804e24203"
server_key_version: "encryptionkey@0"
I20260808 15:52:02.395207 5971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "46974c6e24c248f4ae385d6a28257046"
format_stamp: "Formatted at 2026-08-08 15:52:02 on dist-test-slave-sb2z"
server_key: "bd9a7a4954226e12b3b609c6b408c2c6"
server_key_iv: "3051e7d524290afdf0520c5804e24203"
server_key_version: "encryptionkey@0"
I20260808 15:52:02.398909 5971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:52:02.401273 5990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:02.402395 5971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:52:02.402518 5971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "46974c6e24c248f4ae385d6a28257046"
format_stamp: "Formatted at 2026-08-08 15:52:02 on dist-test-slave-sb2z"
server_key: "bd9a7a4954226e12b3b609c6b408c2c6"
server_key_iv: "3051e7d524290afdf0520c5804e24203"
server_key_version: "encryptionkey@0"
I20260808 15:52:02.402621 5971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:02.423252 5971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:02.424033 5971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:02.424242 5971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:02.431183 5971 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:37013
I20260808 15:52:02.431203 6052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:37013 every 8 connection(s)
I20260808 15:52:02.432153 5971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:02.433323 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 5971
I20260808 15:52:02.433461 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:02.433751 31025 external_mini_cluster.cc:1428] Setting key 97b050637e084438999c23ec9e22e8ec
I20260808 15:52:02.435768 6053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:02.442097 6053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046: Bootstrap starting.
I20260808 15:52:02.442817 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:02.435124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.30.76.126:37013)
0808 15:52:02.435774 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:02.435790 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:02.435846 (+ 56us) server_negotiation.cc:407] Connection header received
0808 15:52:02.436506 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:02.436521 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:02.436885 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:02.437152 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:02.437471 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.438407 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.439621 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.440085 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.440226 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.441119 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:02.441133 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:02.441145 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:02.441503 (+ 358us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:02.441557 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:02.441575 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:02.441648 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:02.441824 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 15:52:02.442107 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":139,"threads_started":1}
I20260808 15:52:02.444561 6053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:02.445495 6053 log.cc:824] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:02.445823 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:52:02.447842 6053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046: No bootstrap required, opened a new log
I20260808 15:52:02.451500 6053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46974c6e24c248f4ae385d6a28257046" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37013 } }
I20260808 15:52:02.451813 6053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:02.451898 6053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46974c6e24c248f4ae385d6a28257046, State: Initialized, Role: FOLLOWER
I20260808 15:52:02.452349 6053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46974c6e24c248f4ae385d6a28257046" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37013 } }
I20260808 15:52:02.452504 6053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:02.452592 6053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:02.452692 6053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:02.453557 6053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46974c6e24c248f4ae385d6a28257046" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37013 } }
I20260808 15:52:02.453893 6053 leader_election.cc:302] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46974c6e24c248f4ae385d6a28257046; no voters:
I20260808 15:52:02.454200 6053 leader_election.cc:288] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:02.454334 6063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:02.454573 6063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [term 1 LEADER]: Becoming Leader. State: Replica: 46974c6e24c248f4ae385d6a28257046, State: Running, Role: LEADER
I20260808 15:52:02.454872 6063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46974c6e24c248f4ae385d6a28257046" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37013 } }
I20260808 15:52:02.455387 6053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:02.456512 6064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46974c6e24c248f4ae385d6a28257046" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46974c6e24c248f4ae385d6a28257046" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37013 } } }
I20260808 15:52:02.456475 6066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46974c6e24c248f4ae385d6a28257046. Latest consensus state: current_term: 1 leader_uuid: "46974c6e24c248f4ae385d6a28257046" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46974c6e24c248f4ae385d6a28257046" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 37013 } } }
I20260808 15:52:02.456622 6064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:02.456622 6066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:02.457031 6079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:02.460162 6079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:02.465580 6079 catalog_manager.cc:1347] Generated new cluster ID: 161a8b1598134302970519ce8b2b8293
I20260808 15:52:02.465648 6079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:02.480273 6079 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:02.481249 6079 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:02.488153 6079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46974c6e24c248f4ae385d6a28257046: Generated new TSK 0
I20260808 15:52:02.488772 6079 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 15:52:02.521744 6103 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 15:52:02.521836 6103 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 15:52:02.522549 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:02.507237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.30.76.126:37013)
0808 15:52:02.507369 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:02.507373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:02.507548 (+ 175us) server_negotiation.cc:407] Connection header received
0808 15:52:02.507752 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:02.507755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:02.507802 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:02.507913 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:52:02.508153 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.508693 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.509561 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.509784 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.509891 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.510206 (+ 315us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:52:02.522132 (+ 11926us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46507/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 15:52:02.522400 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43834: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46507/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 15:52:02.522635 6101 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:02.507166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:37013 (local address 127.0.0.1:43834)
0808 15:52:02.507451 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:02.507463 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:02.507609 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:02.507912 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:02.507916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:02.507924 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:52:02.508036 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:02.508042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.508806 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.508810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:02.509433 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:02.509441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.509913 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.509917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:02.510010 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.510031 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:52:02.522309 (+ 12278us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46507/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 15:52:02.522470 (+ 161us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.76.126:37013: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46507/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
W20260808 15:52:02.522624 6059 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43834: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46507/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260808 15:52:02.523401 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 5971
2026-08-08T15:52:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 15:52:02.573055 31025 mini_oidc.cc:152] Starting JWKS server
I20260808 15:52:02.573882 31025 webserver.cc:460] Webserver started at https://127.0.0.1:35933/ using document root <none> and password file <none>
I20260808 15:52:02.573966 31025 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 15:52:02.574229 31025 webserver.cc:460] Webserver started at http://127.0.0.1:34513/ using document root <none> and password file <none>
2026-08-08T15:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:02Z Disabled control of system clock
I20260808 15:52:02.584357 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39917
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39917
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35933/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 15:52:02.695160 6109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:02.695400 6109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:02.695436 6109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:02.699229 6109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:02.699277 6109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:02.699292 6109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:02.699308 6109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:02.699324 6109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:52:02.699393 6109 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:52:02.699422 6109 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35933/jwks.json
I20260808 15:52:02.704129 6109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35933/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:02.705269 6109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:02.706681 6109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:02.715257 6115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:02.715286 6116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:02.715303 6118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:02.715652 6109 server_base.cc:1034] running on GCE node
I20260808 15:52:02.716347 6109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:02.717464 6109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:02.718634 6109 hybrid_clock.cc:648] HybridClock initialized: now 1786204322718608 us; error 31 us; skew 500 ppm
I20260808 15:52:02.720593 6109 webserver.cc:460] Webserver started at http://127.30.76.126:34751/ using document root <none> and password file <none>
I20260808 15:52:02.721216 6109 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:02.721312 6109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:02.721544 6109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:02.723650 6109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "9a02eb5cdb7a4265a32bb893bde75d7b"
format_stamp: "Formatted at 2026-08-08 15:52:02 on dist-test-slave-sb2z"
server_key: "35000d685c4c877c41cb4b8b4f91b6fa"
server_key_iv: "8fff2ad3b8ef475ba3791ed08613a362"
server_key_version: "encryptionkey@0"
I20260808 15:52:02.724222 6109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "9a02eb5cdb7a4265a32bb893bde75d7b"
format_stamp: "Formatted at 2026-08-08 15:52:02 on dist-test-slave-sb2z"
server_key: "35000d685c4c877c41cb4b8b4f91b6fa"
server_key_iv: "8fff2ad3b8ef475ba3791ed08613a362"
server_key_version: "encryptionkey@0"
I20260808 15:52:02.727836 6109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:02.730155 6125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:02.731346 6109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:02.731483 6109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "9a02eb5cdb7a4265a32bb893bde75d7b"
format_stamp: "Formatted at 2026-08-08 15:52:02 on dist-test-slave-sb2z"
server_key: "35000d685c4c877c41cb4b8b4f91b6fa"
server_key_iv: "8fff2ad3b8ef475ba3791ed08613a362"
server_key_version: "encryptionkey@0"
I20260808 15:52:02.731638 6109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:02.759474 6109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:02.760255 6109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:02.760465 6109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:02.767596 6109 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39917
I20260808 15:52:02.767575 6194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39917 every 8 connection(s)
I20260808 15:52:02.768560 6109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:02.770687 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 6109
I20260808 15:52:02.770818 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:02.771142 31025 external_mini_cluster.cc:1428] Setting key 1f2a27427666ad566be161a165bb9cd0
I20260808 15:52:02.772266 6195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:02.778036 6195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b: Bootstrap starting.
I20260808 15:52:02.780330 6195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:02.780355 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:02.772558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.30.76.126:39917)
0808 15:52:02.773227 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:02.773242 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:02.773278 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:52:02.773905 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:02.773921 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:02.774343 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:02.774728 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:02.775142 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.776195 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.777108 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.777602 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.777769 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.778710 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:02.778715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:02.778727 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:02.779106 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:02.779161 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:02.779180 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:02.779289 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:02.779405 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 15:52:02.779681 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":132,"threads_started":1}
I20260808 15:52:02.781291 6195 log.cc:824] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:02.783191 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:52:02.783499 6195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b: No bootstrap required, opened a new log
I20260808 15:52:02.787029 6195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39917 } }
I20260808 15:52:02.787247 6195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:02.787324 6195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a02eb5cdb7a4265a32bb893bde75d7b, State: Initialized, Role: FOLLOWER
I20260808 15:52:02.787798 6195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39917 } }
I20260808 15:52:02.787920 6195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:02.788014 6195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:02.788122 6195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:02.789016 6195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39917 } }
I20260808 15:52:02.789345 6195 leader_election.cc:302] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a02eb5cdb7a4265a32bb893bde75d7b; no voters:
I20260808 15:52:02.789674 6195 leader_election.cc:288] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:02.789783 6203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:02.790017 6203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [term 1 LEADER]: Becoming Leader. State: Replica: 9a02eb5cdb7a4265a32bb893bde75d7b, State: Running, Role: LEADER
I20260808 15:52:02.790324 6203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39917 } }
I20260808 15:52:02.790930 6195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:02.791558 6206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a02eb5cdb7a4265a32bb893bde75d7b. Latest consensus state: current_term: 1 leader_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39917 } } }
I20260808 15:52:02.791703 6206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:02.792140 6204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a02eb5cdb7a4265a32bb893bde75d7b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39917 } } }
I20260808 15:52:02.792258 6204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:02.792433 6213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:02.795567 6213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:02.800915 6213 catalog_manager.cc:1347] Generated new cluster ID: e859513e18e043269662b72b0e01054d
I20260808 15:52:02.800990 6213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:02.831838 6213 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:02.833032 6213 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:02.848485 6213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a02eb5cdb7a4265a32bb893bde75d7b: Generated new TSK 0
I20260808 15:52:02.849282 6213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:52:02.867877 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:02.863201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34020 (local address 127.30.76.126:39917)
0808 15:52:02.863347 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:02.863355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:02.863570 (+ 215us) server_negotiation.cc:407] Connection header received
0808 15:52:02.863779 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:02.863781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:02.863829 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:02.863937 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:52:02.864179 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.864675 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.865580 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.865792 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.865902 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.866213 (+ 311us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:52:02.867493 (+ 1280us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 15:52:02.867754 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34020: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 15:52:02.867940 6198 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34020: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 15:52:02.869756 6237 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:02.863113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39917 (local address 127.0.0.1:34020)
0808 15:52:02.863465 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:02.863476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:02.863635 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:02.863936 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:02.863939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:02.863947 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:52:02.864056 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:02.864071 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.864813 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.864817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:02.865444 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:02.865452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:02.865910 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:02.865914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:02.866008 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:02.866029 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:52:02.867694 (+ 1665us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 15:52:02.869578 (+ 1884us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.76.126:39917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":1674,"thread_start_us":107,"threads_started":1}
I20260808 15:52:02.870456 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 6109
E20260808 15:52:02.886461 6238 webserver.cc:533] Webserver: sslize failed:
E20260808 15:52:02.886539 6238 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T15:52:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (346 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T15:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:02Z Disabled control of system clock
I20260808 15:52:02.929490 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45975
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45975
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/testchainca.cert with env {}
W20260808 15:52:03.043774 6243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:03.044024 6243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:03.044065 6243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:03.048305 6243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:03.048367 6243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:03.048383 6243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:03.048400 6243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:03.048416 6243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:52:03.048434 6243 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:52:03.048449 6243 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 15:52:03.053791 6243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:03.054853 6243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:03.055862 6243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:03.063853 6254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:03.063845 6250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:03.063844 6249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:03.064353 6243 server_base.cc:1034] running on GCE node
I20260808 15:52:03.064939 6243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:03.065936 6243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:03.067121 6243 hybrid_clock.cc:648] HybridClock initialized: now 1786204323067110 us; error 25 us; skew 500 ppm
I20260808 15:52:03.068861 6243 webserver.cc:460] Webserver started at http://127.30.76.126:38445/ using document root <none> and password file <none>
I20260808 15:52:03.069422 6243 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:03.069514 6243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:03.069741 6243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:03.071902 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "f1d223ad88a7418d93bad8cb6f195755"
format_stamp: "Formatted at 2026-08-08 15:52:03 on dist-test-slave-sb2z"
server_key: "3afdd5f9b177b60f19e9c4cadd15cecd"
server_key_iv: "32ef0076b1008d08b7b66638e1a8f720"
server_key_version: "encryptionkey@0"
I20260808 15:52:03.072499 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "f1d223ad88a7418d93bad8cb6f195755"
format_stamp: "Formatted at 2026-08-08 15:52:03 on dist-test-slave-sb2z"
server_key: "3afdd5f9b177b60f19e9c4cadd15cecd"
server_key_iv: "32ef0076b1008d08b7b66638e1a8f720"
server_key_version: "encryptionkey@0"
I20260808 15:52:03.076066 6243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:52:03.078281 6263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:03.079478 6243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:03.079630 6243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "f1d223ad88a7418d93bad8cb6f195755"
format_stamp: "Formatted at 2026-08-08 15:52:03 on dist-test-slave-sb2z"
server_key: "3afdd5f9b177b60f19e9c4cadd15cecd"
server_key_iv: "32ef0076b1008d08b7b66638e1a8f720"
server_key_version: "encryptionkey@0"
I20260808 15:52:03.079722 6243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:03.096858 6243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:03.097450 6243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:03.097592 6243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:03.104089 6243 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45975
I20260808 15:52:03.104095 6333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45975 every 8 connection(s)
I20260808 15:52:03.105072 6243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:03.105392 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 6243
I20260808 15:52:03.105530 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:03.105870 31025 external_mini_cluster.cc:1428] Setting key 10d7ffd39b5d9c2533c3eee0f73fe4e7
I20260808 15:52:03.108373 6334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:03.114466 6334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755: Bootstrap starting.
I20260808 15:52:03.114853 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:03.107227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.30.76.126:45975)
0808 15:52:03.107844 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:03.107860 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:03.107894 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:52:03.108543 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:03.108561 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:03.108985 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:03.109252 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:03.109573 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:03.110505 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:03.111710 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:03.112148 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:03.112263 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:03.113121 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:03.113126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:03.113138 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:03.113503 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:03.113556 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:03.113576 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:03.113657 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:03.113830 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 15:52:03.114124 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":129,"threads_started":1}
I20260808 15:52:03.116945 6334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:03.117785 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:52:03.117815 6334 log.cc:824] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:03.120124 6334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755: No bootstrap required, opened a new log
I20260808 15:52:03.123296 6334 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d223ad88a7418d93bad8cb6f195755" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45975 } }
I20260808 15:52:03.123518 6334 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:03.123617 6334 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d223ad88a7418d93bad8cb6f195755, State: Initialized, Role: FOLLOWER
I20260808 15:52:03.124049 6334 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d223ad88a7418d93bad8cb6f195755" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45975 } }
I20260808 15:52:03.124197 6334 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:03.124284 6334 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:03.124382 6334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:03.125330 6334 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d223ad88a7418d93bad8cb6f195755" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45975 } }
I20260808 15:52:03.125686 6334 leader_election.cc:302] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d223ad88a7418d93bad8cb6f195755; no voters:
I20260808 15:52:03.125990 6334 leader_election.cc:288] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:03.126168 6341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:03.126437 6341 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [term 1 LEADER]: Becoming Leader. State: Replica: f1d223ad88a7418d93bad8cb6f195755, State: Running, Role: LEADER
I20260808 15:52:03.126729 6341 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d223ad88a7418d93bad8cb6f195755" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45975 } }
I20260808 15:52:03.127213 6334 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:03.128376 6344 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1d223ad88a7418d93bad8cb6f195755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d223ad88a7418d93bad8cb6f195755" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45975 } } }
I20260808 15:52:03.128523 6344 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:03.128821 6346 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d223ad88a7418d93bad8cb6f195755. Latest consensus state: current_term: 1 leader_uuid: "f1d223ad88a7418d93bad8cb6f195755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d223ad88a7418d93bad8cb6f195755" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45975 } } }
I20260808 15:52:03.128922 6346 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:03.129019 6360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:03.132037 6360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:03.137636 6360 catalog_manager.cc:1347] Generated new cluster ID: 8294172db0c14614bed85b6249d82c38
I20260808 15:52:03.137705 6360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:03.156385 6360 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:03.157518 6360 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:03.163199 6360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1d223ad88a7418d93bad8cb6f195755: Generated new TSK 0
I20260808 15:52:03.163934 6360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:52:03.218153 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:03.177871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.30.76.126:45975)
0808 15:52:03.178004 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:03.178009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:03.178193 (+ 184us) server_negotiation.cc:407] Connection header received
0808 15:52:03.178357 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:03.178359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:03.178404 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:03.178480 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:52:03.178741 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:03.179231 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:03.180468 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:03.180697 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:03.180776 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:03.181130 (+ 354us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:52:03.217760 (+ 36630us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 15:52:03.218029 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51304: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 15:52:03.218258 6338 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51304: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 15:52:03.218430 6378 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:03.177802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45975 (local address 127.0.0.1:51304)
0808 15:52:03.178085 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:03.178100 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:03.178228 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:03.178508 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:03.178511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:03.178519 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:52:03.178632 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:03.178638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:03.179338 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:03.179345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:03.180355 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:03.180366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:03.180824 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:03.180828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:03.180936 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:03.180964 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:52:03.218006 (+ 37042us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 15:52:03.218244 (+ 238us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.76.126:45975: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":121,"threads_started":1}
I20260808 15:52:03.219051 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 6243
2026-08-08T15:52:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (325 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:03 dist-test-slave-sb2z krb5kdc[6394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:03 dist-test-slave-sb2z krb5kdc[6394](info): set up 2 sockets
Aug 08 15:52:03 dist-test-slave-sb2z krb5kdc[6394](info): commencing operation
krb5kdc: starting...
W20260808 15:52:04.698832 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:04 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:52:04.826203 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39463
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:04.939620 6416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:04.939921 6416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:04.939996 6416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:04.945453 6416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:04.945528 6416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:04.945564 6416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:04.945621 6416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:04.945657 6416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:04.950867 6416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:04.951951 6416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:04.952991 6416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:04.961449 6422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:04.961577 6421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:04.961475 6424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:04.961899 6416 server_base.cc:1034] running on GCE node
I20260808 15:52:04.962481 6416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:04.963622 6416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:04.964811 6416 hybrid_clock.cc:648] HybridClock initialized: now 1786204324964801 us; error 27 us; skew 500 ppm
Aug 08 15:52:04 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204324, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:04.971453 6416 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:52:04.972321 6416 webserver.cc:460] Webserver started at http://127.30.76.126:36051/ using document root <none> and password file <none>
I20260808 15:52:04.972896 6416 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:04.972988 6416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:04.973222 6416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:04.975211 6416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "eace56a382fe4cd6a56d076c6d3f506b"
format_stamp: "Formatted at 2026-08-08 15:52:04 on dist-test-slave-sb2z"
server_key: "08c933b2ae1dea9d073f5b764090bd89"
server_key_iv: "e8885ff5e316d07b705fafaa0023b292"
server_key_version: "encryptionkey@0"
I20260808 15:52:04.975835 6416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "eace56a382fe4cd6a56d076c6d3f506b"
format_stamp: "Formatted at 2026-08-08 15:52:04 on dist-test-slave-sb2z"
server_key: "08c933b2ae1dea9d073f5b764090bd89"
server_key_iv: "e8885ff5e316d07b705fafaa0023b292"
server_key_version: "encryptionkey@0"
I20260808 15:52:04.979278 6416 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:52:04.981597 6435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:04.982754 6416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:04.982896 6416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "eace56a382fe4cd6a56d076c6d3f506b"
format_stamp: "Formatted at 2026-08-08 15:52:04 on dist-test-slave-sb2z"
server_key: "08c933b2ae1dea9d073f5b764090bd89"
server_key_iv: "e8885ff5e316d07b705fafaa0023b292"
server_key_version: "encryptionkey@0"
I20260808 15:52:04.983006 6416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:04.995383 6416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:04.998319 6416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:04.998517 6416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:05.005929 6416 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39463
I20260808 15:52:05.005959 6506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39463 every 8 connection(s)
I20260808 15:52:05.007180 6416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:05.010300 6507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:05.012403 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 6416
I20260808 15:52:05.012543 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:05.012840 31025 external_mini_cluster.cc:1428] Setting key 22e319988437c0b72d15715c6aba97a3
I20260808 15:52:05.016475 6507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b: Bootstrap starting.
I20260808 15:52:05.018971 6507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:05.019841 6507 log.cc:824] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:05.022109 6507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b: No bootstrap required, opened a new log
I20260808 15:52:05.025272 6507 raft_consensus.cc:348] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace56a382fe4cd6a56d076c6d3f506b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39463 } }
I20260808 15:52:05.025516 6507 raft_consensus.cc:374] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:05.025586 6507 raft_consensus.cc:729] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eace56a382fe4cd6a56d076c6d3f506b, State: Initialized, Role: FOLLOWER
I20260808 15:52:05.026036 6507 consensus_queue.cc:260] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace56a382fe4cd6a56d076c6d3f506b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39463 } }
I20260808 15:52:05.026190 6507 raft_consensus.cc:388] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:05.026280 6507 raft_consensus.cc:482] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:05.026399 6507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:05.027379 6507 raft_consensus.cc:504] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace56a382fe4cd6a56d076c6d3f506b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39463 } }
I20260808 15:52:05.027743 6507 leader_election.cc:302] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eace56a382fe4cd6a56d076c6d3f506b; no voters:
I20260808 15:52:05.027906 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.014273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.30.76.126:39463)
0808 15:52:05.014671 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.014693 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.014727 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:52:05.015447 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.015466 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.015832 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.016099 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.017004 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.017964 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.019162 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.019724 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.019843 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.022596 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.022601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.022614 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.022641 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.024899 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.025345 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.025351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.025356 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.025463 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.025788 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.025792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.025796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.026300 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.026520 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.026822 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.027108 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":132,"threads_started":1}
I20260808 15:52:05.028090 6507 leader_election.cc:288] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:05.028221 6516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:05.028436 6516 raft_consensus.cc:686] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [term 1 LEADER]: Becoming Leader. State: Replica: eace56a382fe4cd6a56d076c6d3f506b, State: Running, Role: LEADER
I20260808 15:52:05.028772 6516 consensus_queue.cc:237] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace56a382fe4cd6a56d076c6d3f506b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39463 } }
I20260808 15:52:05.029376 6507 sys_catalog.cc:564] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:05.031114 6516 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eace56a382fe4cd6a56d076c6d3f506b. Latest consensus state: current_term: 1 leader_uuid: "eace56a382fe4cd6a56d076c6d3f506b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace56a382fe4cd6a56d076c6d3f506b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39463 } } }
I20260808 15:52:05.031262 6516 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:05.031780 6518 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eace56a382fe4cd6a56d076c6d3f506b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace56a382fe4cd6a56d076c6d3f506b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39463 } } }
I20260808 15:52:05.031981 6518 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b [sys.catalog]: This master's current role is: LEADER
W20260808 15:52:05.034576 6537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:52:05.034648 6537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:52:05.034729 6536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:05.035452 6536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:05.040935 6536 catalog_manager.cc:1347] Generated new cluster ID: 041ad019254b49238c96ef4d86efbe55
I20260808 15:52:05.041000 6536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:05.060361 6536 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:05.061681 6536 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:05.069181 6536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eace56a382fe4cd6a56d076c6d3f506b: Generated new TSK 0
I20260808 15:52:05.069851 6536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:52:05.111236 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39463
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:05.238368 6544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:05.238636 6544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:05.238721 6544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:05.242808 6544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:05.242865 6544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:05.242921 6544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:05.248118 6544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:05.249159 6544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:05.250229 6544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:05.259812 6555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:05.259950 6544 server_base.cc:1034] running on GCE node
W20260808 15:52:05.260039 6549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:05.259838 6550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:05.260485 6544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:05.261168 6544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:05.262342 6544 hybrid_clock.cc:648] HybridClock initialized: now 1786204325262306 us; error 46 us; skew 500 ppm
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.265280 6544 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:52:05.266145 6544 webserver.cc:460] Webserver started at http://127.30.76.65:44115/ using document root <none> and password file <none>
I20260808 15:52:05.266703 6544 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:05.266801 6544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:05.267042 6544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:05.269088 6544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "1b8531c798694b4a88c394b1c3104b50"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "dad13d09a30fec57442a0bdeb23b3468"
server_key_iv: "fb3a7fa90f788c0de415d68b7af495f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.269649 6544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "1b8531c798694b4a88c394b1c3104b50"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "dad13d09a30fec57442a0bdeb23b3468"
server_key_iv: "fb3a7fa90f788c0de415d68b7af495f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.273234 6544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:52:05.275667 6566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.276806 6544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:05.276959 6544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "1b8531c798694b4a88c394b1c3104b50"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "dad13d09a30fec57442a0bdeb23b3468"
server_key_iv: "fb3a7fa90f788c0de415d68b7af495f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.277071 6544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:05.289883 6544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:05.293004 6544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:05.293200 6544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:05.293759 6544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:05.294530 6544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:05.294601 6544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.294684 6544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:05.294740 6544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.304114 6544 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:43079
I20260808 15:52:05.304183 6720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:43079 every 8 connection(s)
I20260808 15:52:05.304972 6544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:05.308879 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 6544
I20260808 15:52:05.309012 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:05.309387 31025 external_mini_cluster.cc:1428] Setting key f0fb17238925c67d6e0021f498111e42
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:05.320910 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.306820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:43471 (local address 127.30.76.126:39463)
0808 15:52:05.306979 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.306984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.307843 (+ 859us) server_negotiation.cc:407] Connection header received
0808 15:52:05.309245 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.309250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.309311 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.309407 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.310696 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.311214 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.312397 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.312790 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.312877 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.316075 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.316081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.316088 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.316119 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.317934 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.318703 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.318706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.318708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.318833 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.319298 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.319302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.319304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.319609 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.319718 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.320491 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.320682 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:52:05.321870 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.307185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39463 (local address 127.30.76.65:43471)
0808 15:52:05.307703 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:05.307746 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:05.308995 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:05.309595 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:05.309601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:05.309995 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:05.310513 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:05.310534 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.311344 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.311346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:05.312274 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:05.312284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.312964 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.312971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:05.313199 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.313802 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:05.313827 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:05.315878 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:05.318075 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:05.318085 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:05.318099 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:05.318524 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:05.318944 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:05.318947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:05.318949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:05.319183 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:05.319742 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:05.319748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:05.320107 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 15:52:05.320383 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 15:52:05.320698 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260808 15:52:05.323376 6723 heartbeater.cc:347] Connected to a master server at 127.30.76.126:39463
I20260808 15:52:05.323668 6723 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:05.324342 6723 heartbeater.cc:511] Master 127.30.76.126:39463 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:52:05.326637 6458 ts_manager.cc:194] Registered new tserver with Master: 1b8531c798694b4a88c394b1c3104b50 (127.30.76.65:43079)
I20260808 15:52:05.328228 6458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:43471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:52:05.342023 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39463
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:05.453806 6729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:05.454051 6729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:05.454093 6729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:05.458319 6729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:05.458379 6729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:05.458436 6729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:05.463466 6729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:05.464442 6729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:05.465363 6729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:05.474514 6739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:05.474532 6743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:05.474524 6741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:05.474826 6729 server_base.cc:1034] running on GCE node
I20260808 15:52:05.475214 6729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:05.475919 6729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:05.477106 6729 hybrid_clock.cc:648] HybridClock initialized: now 1786204325477090 us; error 28 us; skew 500 ppm
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.479880 6729 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:52:05.480762 6729 webserver.cc:460] Webserver started at http://127.30.76.66:40193/ using document root <none> and password file <none>
I20260808 15:52:05.481328 6729 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:05.481420 6729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:05.481657 6729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:05.483742 6729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "df08c133763d435f9bd2288d725e4d85"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "497145300494ffaf3ba2260493a73543"
server_key_iv: "c6470af27cc74ca3933fbfa8db588358"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.484360 6729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "df08c133763d435f9bd2288d725e4d85"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "497145300494ffaf3ba2260493a73543"
server_key_iv: "c6470af27cc74ca3933fbfa8db588358"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.487920 6729 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:05.490113 6755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.491250 6729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:05.491382 6729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "df08c133763d435f9bd2288d725e4d85"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "497145300494ffaf3ba2260493a73543"
server_key_iv: "c6470af27cc74ca3933fbfa8db588358"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.491479 6729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:05.501874 6729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:05.504683 6729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:05.504879 6729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:05.505460 6729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:05.506265 6729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:05.506313 6729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.506381 6729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:05.506420 6729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.515456 6729 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:42609
I20260808 15:52:05.515481 6907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:42609 every 8 connection(s)
I20260808 15:52:05.516377 6729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:05.518726 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 6729
I20260808 15:52:05.518867 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:05.519174 31025 external_mini_cluster.cc:1428] Setting key 635b6f1a2ebed58511880c2eb98d1f69
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:52:05.531324 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.518114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:51227 (local address 127.30.76.126:39463)
0808 15:52:05.518285 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.518290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.519191 (+ 901us) server_negotiation.cc:407] Connection header received
0808 15:52:05.520795 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.520798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.520854 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.520955 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.522088 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.522900 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.524118 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.524348 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.524462 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.527260 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.527268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.527272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.527305 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.528981 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.529498 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.529502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.529504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.529597 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.529998 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.530001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.530003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.530320 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.530416 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.530962 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.531200 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:52:05.532287 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.518562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39463 (local address 127.30.76.66:51227)
0808 15:52:05.519023 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:05.519073 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:05.520558 (+ 1485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:05.521128 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:05.521135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:05.521478 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:05.521940 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:05.521953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.523046 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.523049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:05.523978 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:05.523991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.524501 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.524506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:05.524678 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.525348 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:05.525374 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:05.527068 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:05.529101 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:05.529106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:05.529117 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:05.529370 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:05.529710 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:05.529716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:05.529721 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:05.529864 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:05.530416 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:05.530421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:05.530712 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 15:52:05.530912 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 15:52:05.531221 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:52:05.533522 6908 heartbeater.cc:347] Connected to a master server at 127.30.76.126:39463
I20260808 15:52:05.533785 6908 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:05.534278 6908 heartbeater.cc:511] Master 127.30.76.126:39463 requested a full tablet report, sending...
I20260808 15:52:05.535554 6458 ts_manager.cc:194] Registered new tserver with Master: df08c133763d435f9bd2288d725e4d85 (127.30.76.66:42609)
I20260808 15:52:05.536342 6458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:51227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:52:05.550580 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39463
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:05.664371 6916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:05.664604 6916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:05.664667 6916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:05.668643 6916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:05.668694 6916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:05.668795 6916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:05.673882 6916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:39463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:05.674849 6916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:05.675889 6916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:05.685425 6926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:05.685438 6924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:05.685585 6916 server_base.cc:1034] running on GCE node
W20260808 15:52:05.685493 6921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:05.686132 6916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:05.686705 6916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:05.687889 6916 hybrid_clock.cc:648] HybridClock initialized: now 1786204325687879 us; error 30 us; skew 500 ppm
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.690531 6916 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:52:05.691399 6916 webserver.cc:460] Webserver started at http://127.30.76.67:39689/ using document root <none> and password file <none>
I20260808 15:52:05.692016 6916 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:05.692139 6916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:05.692380 6916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:05.694430 6916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "1812889d20034ae7b50617e9a975bca6"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "e83692e22d48ce28bde7e974b168b494"
server_key_iv: "10305a9583659dbe27aff45b2c3a3fb2"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.695001 6916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "1812889d20034ae7b50617e9a975bca6"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "e83692e22d48ce28bde7e974b168b494"
server_key_iv: "10305a9583659dbe27aff45b2c3a3fb2"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.698630 6916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:52:05.701110 6937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.702275 6916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:05.702409 6916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "1812889d20034ae7b50617e9a975bca6"
format_stamp: "Formatted at 2026-08-08 15:52:05 on dist-test-slave-sb2z"
server_key: "e83692e22d48ce28bde7e974b168b494"
server_key_iv: "10305a9583659dbe27aff45b2c3a3fb2"
server_key_version: "encryptionkey@0"
I20260808 15:52:05.702517 6916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:05.712951 6916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:05.715775 6916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:05.715965 6916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:05.716521 6916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:05.717324 6916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:05.717396 6916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.717464 6916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:05.717511 6916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:05.726230 6916 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:35339
I20260808 15:52:05.726261 7090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:35339 every 8 connection(s)
I20260808 15:52:05.727077 6916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:05.737177 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 6916
I20260808 15:52:05.737315 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:05.737599 31025 external_mini_cluster.cc:1428] Setting key c21cb8c80762e40297cdc35e9b429ebe
I20260808 15:52:05.740682 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.728800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37339 (local address 127.30.76.126:39463)
0808 15:52:05.728954 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.728959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.729932 (+ 973us) server_negotiation.cc:407] Connection header received
0808 15:52:05.731428 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.731431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.731508 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.731623 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.732689 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.733211 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.734254 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.734459 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.734533 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.736862 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.736865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.736867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.736887 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.738521 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.739178 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.739182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.739183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.739277 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.739640 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.739644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.739645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.739906 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.740024 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.740462 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.740556 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:52:05.741503 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.729223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39463 (local address 127.30.76.67:37339)
0808 15:52:05.729645 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:05.729824 (+ 179us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:05.731242 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:05.731765 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:05.731771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:05.732105 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:05.732564 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:05.732576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.733313 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.733315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:05.734153 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:05.734161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.734581 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.734583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:05.734716 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.735291 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:05.735307 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:05.736718 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:05.738682 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:05.738689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:05.738701 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:05.739060 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:05.739369 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:05.739372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:05.739373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:05.739499 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:05.740024 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:05.740029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:05.740258 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 15:52:05.740413 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 15:52:05.740648 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260808 15:52:05.742550 7091 heartbeater.cc:347] Connected to a master server at 127.30.76.126:39463
I20260808 15:52:05.742789 7091 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:05.743257 7091 heartbeater.cc:511] Master 127.30.76.126:39463 requested a full tablet report, sending...
I20260808 15:52:05.744277 6458 ts_manager.cc:194] Registered new tserver with Master: 1812889d20034ae7b50617e9a975bca6 (127.30.76.67:35339)
I20260808 15:52:05.744838 6458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:37339
I20260808 15:52:05.751858 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/test-admin.keytab.
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.768108 31025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 15:52:05.771004 7099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:05.769458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.30.76.65:43079)
0808 15:52:05.769842 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.769854 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.769876 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:52:05.769925 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.769930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.770111 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.770268 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.770772 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38494: BlockingRecv error: recv got EOF from 127.0.0.1:38494 (error 108)
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":134,"threads_started":1}
W20260808 15:52:05.771026 31025 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc' of type FILE
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.772145 31025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.773056 31025 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:52:05.782740 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.773909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.30.76.65:43079)
0808 15:52:05.774040 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.774045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.774131 (+ 86us) server_negotiation.cc:407] Connection header received
0808 15:52:05.774289 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.774292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.774341 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.774420 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.774934 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.775808 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.776655 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.777036 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.777151 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.779383 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.779392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.779405 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.779427 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.780957 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.781408 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.781416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.781418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.781499 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.781779 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.781782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.781784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.782016 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.782222 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.782439 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.782599 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 15:52:05.786262 7099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:05.785396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.30.76.65:43079)
0808 15:52:05.785530 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.785533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.785549 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:05.785668 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.785670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.785705 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.785785 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.786174 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38512: BlockingRecv error: recv got EOF from 127.0.0.1:38512 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 15:52:05.786574 31025 init.cc:244] Error while opening credential cache '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc)
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.787685 31025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.788544 31025 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:52:05.796464 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.789289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.30.76.65:43079)
0808 15:52:05.789406 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.789410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.789507 (+ 97us) server_negotiation.cc:407] Connection header received
0808 15:52:05.789665 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.789667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.789710 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.789790 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.790250 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.790740 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.791504 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.791795 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.791903 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.793893 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.793896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.793898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.793919 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.795147 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.795513 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.795516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.795517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.795639 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.795879 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.795881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.795882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.796023 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.796113 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.796274 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.796357 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 15:52:05.799049 7099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:05.798051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.30.76.65:43079)
0808 15:52:05.798183 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.798187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.798278 (+ 91us) server_negotiation.cc:407] Connection header received
0808 15:52:05.798429 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.798432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.798468 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.798561 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.798945 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38538: BlockingRecv error: recv got EOF from 127.0.0.1:38538 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 15:52:05.799220 31025 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786204294980517-31025-0/krb5kdc/krb5cc' of type FILE
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.800479 31025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:05.801426 31025 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[6394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:52:05.809430 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:05.802154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.30.76.65:43079)
0808 15:52:05.802251 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:05.802256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:05.802304 (+ 48us) server_negotiation.cc:407] Connection header received
0808 15:52:05.802431 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:05.802434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:05.802474 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:05.802557 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:05.803036 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.803581 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.804343 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:05.804556 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:05.804666 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:05.806726 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:05.806729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:05.806731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:05.806751 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:05.808108 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.808470 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.808472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.808473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.808539 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:05.808761 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:05.808763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:05.808764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:05.808898 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:05.808986 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:05.809170 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 15:52:05.809279 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:52:05.810362 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 6544
I20260808 15:52:05.823242 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 6729
I20260808 15:52:05.836552 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 6916
I20260808 15:52:05.844555 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 6416
2026-08-08T15:52:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2370 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[7114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[7114](info): set up 2 sockets
Aug 08 15:52:05 dist-test-slave-sb2z krb5kdc[7114](info): commencing operation
krb5kdc: starting...
W20260808 15:52:07.126390 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab.
Entry for principal oryx/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab.
I20260808 15:52:07.250293 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45997
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab
--principal=oryx/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:07.368300 7134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:07.368593 7134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:07.368673 7134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:07.372720 7134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:07.372803 7134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:07.372843 7134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:07.372901 7134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:07.372947 7134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:07.378109 7134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab
--principal=oryx/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:07.379159 7134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:07.380239 7134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:07.388595 7142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:07.388610 7143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:07.388739 7134 server_base.cc:1034] running on GCE node
W20260808 15:52:07.388965 7146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:07.389506 7134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:07.390568 7134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:07.391762 7134 hybrid_clock.cc:648] HybridClock initialized: now 1786204327391742 us; error 32 us; skew 500 ppm
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:07.394402 7134 init.cc:377] Logged in from keytab as oryx/127.30.76.126@KRBTEST.COM (short username oryx)
I20260808 15:52:07.395224 7134 webserver.cc:460] Webserver started at http://127.30.76.126:40259/ using document root <none> and password file <none>
I20260808 15:52:07.395821 7134 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:07.395910 7134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:07.396142 7134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:07.398169 7134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "82d5bcb2886711215c71e28f2d4eeeef"
server_key_iv: "c43212b7f7e135819e3273dd1e47c8c8"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.398741 7134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "82d5bcb2886711215c71e28f2d4eeeef"
server_key_iv: "c43212b7f7e135819e3273dd1e47c8c8"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.402300 7134 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:52:07.404789 7159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.405908 7134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:07.406046 7134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "82d5bcb2886711215c71e28f2d4eeeef"
server_key_iv: "c43212b7f7e135819e3273dd1e47c8c8"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.406188 7134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:07.431510 7134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:07.434334 7134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:07.434556 7134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:07.442000 7134 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45997
I20260808 15:52:07.442376 7238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45997 every 8 connection(s)
I20260808 15:52:07.442981 7134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:07.445855 7242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:07.447674 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 7134
I20260808 15:52:07.447809 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:07.448087 31025 external_mini_cluster.cc:1428] Setting key a8ff9698a24d3b0b765bc8a50764c4c5
I20260808 15:52:07.453114 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Bootstrap starting.
I20260808 15:52:07.456138 7242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:07.457000 7242 log.cc:824] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:07.459648 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: No bootstrap required, opened a new log
I20260808 15:52:07.463351 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:07.449496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.30.76.126:45997)
0808 15:52:07.449935 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:07.449956 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:07.450003 (+ 47us) server_negotiation.cc:407] Connection header received
0808 15:52:07.450959 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:07.450988 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:07.451429 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:07.451847 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:07.452498 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.454168 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.454976 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.455731 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.455901 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:07.458660 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:07.458665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:07.458677 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:07.458702 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:07.460471 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:07.461013 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:07.461018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:07.461022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:07.461147 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:07.461463 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:07.461465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:07.461467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:07.461819 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:07.461948 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:07.462338 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 15:52:07.462601 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":106,"threads_started":1}
I20260808 15:52:07.464171 7242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } }
I20260808 15:52:07.464447 7242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:07.464514 7242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ae4ca4d61d4221ab26b3f24e6dd70f, State: Initialized, Role: FOLLOWER
I20260808 15:52:07.465011 7242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } }
I20260808 15:52:07.465178 7242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:07.465247 7242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:07.465349 7242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:07.466384 7242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } }
I20260808 15:52:07.466688 7242 leader_election.cc:302] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37ae4ca4d61d4221ab26b3f24e6dd70f; no voters:
I20260808 15:52:07.467010 7242 leader_election.cc:288] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:07.467085 7247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:07.467300 7247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 LEADER]: Becoming Leader. State: Replica: 37ae4ca4d61d4221ab26b3f24e6dd70f, State: Running, Role: LEADER
I20260808 15:52:07.467640 7247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } }
I20260808 15:52:07.468468 7242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:07.469228 7249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } }
I20260808 15:52:07.469368 7249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:07.469715 7248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ae4ca4d61d4221ab26b3f24e6dd70f. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } }
I20260808 15:52:07.469797 7248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
W20260808 15:52:07.473299 7264 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:52:07.473380 7264 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:52:07.473440 7256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:07.474278 7256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:07.479664 7256 catalog_manager.cc:1347] Generated new cluster ID: 2dd02fd64a094281b6e2bfa962b0b00b
I20260808 15:52:07.479732 7256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:07.494765 7256 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:07.495663 7256 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:07.508810 7256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Generated new TSK 0
I20260808 15:52:07.509521 7256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab.
Entry for principal oryx/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab.
I20260808 15:52:07.543345 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45997
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:07.660905 7274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:07.661185 7274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:07.661226 7274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:07.665328 7274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:07.665393 7274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:07.665453 7274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:07.671084 7274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:07.672078 7274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:07.673292 7274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:07.683426 7282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:07.683434 7286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:07.683434 7281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:07.683799 7274 server_base.cc:1034] running on GCE node
I20260808 15:52:07.684232 7274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:07.684860 7274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:07.686062 7274 hybrid_clock.cc:648] HybridClock initialized: now 1786204327686038 us; error 31 us; skew 500 ppm
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:07.689105 7274 init.cc:377] Logged in from keytab as oryx/127.30.76.65@KRBTEST.COM (short username oryx)
I20260808 15:52:07.690043 7274 webserver.cc:460] Webserver started at http://127.30.76.65:45877/ using document root <none> and password file <none>
I20260808 15:52:07.690552 7274 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:07.690611 7274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:07.690780 7274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:07.692909 7274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "9ef601be8bcf41b887bff3d1e29f1af4"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "996bdd9aee17d6503810a6bd0f0d3d67"
server_key_iv: "cb654fa231625cd2c5ba7beb8aa8ecf9"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.693524 7274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "9ef601be8bcf41b887bff3d1e29f1af4"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "996bdd9aee17d6503810a6bd0f0d3d67"
server_key_iv: "cb654fa231625cd2c5ba7beb8aa8ecf9"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.697345 7274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:52:07.699712 7304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.700984 7274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:07.701120 7274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "9ef601be8bcf41b887bff3d1e29f1af4"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "996bdd9aee17d6503810a6bd0f0d3d67"
server_key_iv: "cb654fa231625cd2c5ba7beb8aa8ecf9"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.701222 7274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:07.710316 7274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:07.713581 7274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:07.713784 7274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:07.714368 7274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:07.715178 7274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:07.715226 7274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.715297 7274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:07.715328 7274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.724455 7274 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:46523
I20260808 15:52:07.724498 7473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:46523 every 8 connection(s)
I20260808 15:52:07.725353 7274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:07.730475 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 7274
I20260808 15:52:07.730582 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:07.730890 31025 external_mini_cluster.cc:1428] Setting key b341f7b0c43dfc7a123a8c972527174d
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:07.739349 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:07.727148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:56017 (local address 127.30.76.126:45997)
0808 15:52:07.727304 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:07.727309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:07.728225 (+ 916us) server_negotiation.cc:407] Connection header received
0808 15:52:07.729566 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:07.729569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:07.729617 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:07.729696 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:07.730957 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.731697 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.732855 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.733087 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.733203 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:07.735920 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:07.735924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:07.735926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:07.735948 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:07.737349 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:07.737860 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:07.737864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:07.737866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:07.737948 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:07.738296 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:07.738300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:07.738302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:07.738477 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:07.738584 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:07.739052 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 15:52:07.739178 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:52:07.740136 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:07.727614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:56017)
0808 15:52:07.728066 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:07.728109 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:07.729369 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:07.729860 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:07.729866 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:07.730256 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:07.730798 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:07.730813 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.731832 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.731836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:07.732723 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:07.732733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.733243 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.733247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:07.733419 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:07.734023 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:07.734039 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:07.735745 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:07.737472 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:07.737478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:07.737491 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:07.737754 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:07.738044 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:07.738048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:07.738050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:07.738196 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:07.738584 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:07.738590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:07.738824 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 15:52:07.739011 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 15:52:07.739325 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for oryx/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:52:07.741389 7474 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45997
I20260808 15:52:07.741628 7474 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:07.742098 7474 heartbeater.cc:511] Master 127.30.76.126:45997 requested a full tablet report, sending...
I20260808 15:52:07.743891 7189 ts_manager.cc:194] Registered new tserver with Master: 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.66@KRBTEST.COM" created.
I20260808 15:52:07.745229 7189 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:56017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab.
Entry for principal oryx/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab.
I20260808 15:52:07.760702 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45997
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:07.874570 7482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:07.874912 7482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:07.874979 7482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:07.879124 7482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:07.879180 7482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:07.879235 7482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:07.884332 7482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:07.885254 7482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:07.886174 7482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:07.895294 7489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:07.895470 7492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:07.895543 7482 server_base.cc:1034] running on GCE node
W20260808 15:52:07.895290 7495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:07.896117 7482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:07.896692 7482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:07.897845 7482 hybrid_clock.cc:648] HybridClock initialized: now 1786204327897808 us; error 45 us; skew 500 ppm
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:07.900619 7482 init.cc:377] Logged in from keytab as oryx/127.30.76.66@KRBTEST.COM (short username oryx)
I20260808 15:52:07.901458 7482 webserver.cc:460] Webserver started at http://127.30.76.66:39591/ using document root <none> and password file <none>
I20260808 15:52:07.901957 7482 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:07.902012 7482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:07.902174 7482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:07.904181 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "413997d0e55e42aa8fc81c653ddd8842"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "383cac8e65fd76785e6f8c7a717506ff"
server_key_iv: "cf27b9cb2b347509ed7c337c85463dac"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.904784 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "413997d0e55e42aa8fc81c653ddd8842"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "383cac8e65fd76785e6f8c7a717506ff"
server_key_iv: "cf27b9cb2b347509ed7c337c85463dac"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.908459 7482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:52:07.911017 7503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.912317 7482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:52:07.912457 7482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "413997d0e55e42aa8fc81c653ddd8842"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "383cac8e65fd76785e6f8c7a717506ff"
server_key_iv: "cf27b9cb2b347509ed7c337c85463dac"
server_key_version: "encryptionkey@0"
I20260808 15:52:07.912564 7482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:07.944809 7482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:07.948213 7482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:07.948428 7482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:07.949051 7482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:07.949887 7482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:07.949959 7482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.950031 7482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:07.950075 7482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:07.958806 7482 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:37185
I20260808 15:52:07.958833 7662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:37185 every 8 connection(s)
I20260808 15:52:07.959693 7482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:07.967826 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 7482
I20260808 15:52:07.967962 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:07.968310 31025 external_mini_cluster.cc:1428] Setting key 121686a44fd75c527445a6505b5f2cd5
Aug 08 15:52:07 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:07.973928 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:07.961404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:39813 (local address 127.30.76.126:45997)
0808 15:52:07.961532 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:07.961536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:07.962297 (+ 761us) server_negotiation.cc:407] Connection header received
0808 15:52:07.963733 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:07.963737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:07.963798 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:07.963900 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:07.965027 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.965559 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.966660 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.966879 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.967011 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:07.969919 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:07.969922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:07.969925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:07.969972 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:07.971735 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:07.972393 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:07.972412 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:07.972414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:07.972514 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:07.972835 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:07.972840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:07.972842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:07.973086 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:07.973192 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:07.973629 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 15:52:07.973768 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:52:07.974875 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:07.961710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:39813)
0808 15:52:07.962140 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:07.962181 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:07.963473 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:07.964067 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:07.964074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:07.964409 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:07.964882 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:07.964895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.965692 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.965699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:07.966532 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:07.966542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:07.967049 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:07.967053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:07.967220 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:07.967865 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:07.967889 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:07.969684 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:07.971856 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:07.971865 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:07.971882 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:07.972277 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:07.972611 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:07.972613 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:07.972614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:07.972745 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:07.973190 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:07.973194 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:07.973402 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 15:52:07.973547 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 15:52:07.973826 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":117,"threads_started":1}
I20260808 15:52:07.976151 7663 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45997
I20260808 15:52:07.976439 7663 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:07.977056 7663 heartbeater.cc:511] Master 127.30.76.126:45997 requested a full tablet report, sending...
I20260808 15:52:07.978245 7189 ts_manager.cc:194] Registered new tserver with Master: 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185)
I20260808 15:52:07.979077 7189 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:39813
WARNING: no policy specified for oryx/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab.
Entry for principal oryx/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab.
I20260808 15:52:07.999305 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45997
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:08.113888 7669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:08.114176 7669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:08.114257 7669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:08.118382 7669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:08.118460 7669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:08.118541 7669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:08.123711 7669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:08.124733 7669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:08.125808 7669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:08.135495 7678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.135505 7674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.135488 7676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:08.135710 7669 server_base.cc:1034] running on GCE node
I20260808 15:52:08.136157 7669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:08.136725 7669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:08.137897 7669 hybrid_clock.cc:648] HybridClock initialized: now 1786204328137864 us; error 43 us; skew 500 ppm
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:08.140713 7669 init.cc:377] Logged in from keytab as oryx/127.30.76.67@KRBTEST.COM (short username oryx)
I20260808 15:52:08.141506 7669 webserver.cc:460] Webserver started at http://127.30.76.67:38929/ using document root <none> and password file <none>
I20260808 15:52:08.142011 7669 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.142069 7669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.142235 7669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:08.144228 7669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "ea97b41c75344804ae9678ca333e2d65"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "7c6a0b3db28be844b3cab6f2da2de9f4"
server_key_iv: "090ff1b68f0820d16eb66080c0b38c7e"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.144752 7669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "ea97b41c75344804ae9678ca333e2d65"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "7c6a0b3db28be844b3cab6f2da2de9f4"
server_key_iv: "090ff1b68f0820d16eb66080c0b38c7e"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.148388 7669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:08.150707 7687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.152012 7669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:08.152151 7669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "ea97b41c75344804ae9678ca333e2d65"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "7c6a0b3db28be844b3cab6f2da2de9f4"
server_key_iv: "090ff1b68f0820d16eb66080c0b38c7e"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.152264 7669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.170291 7669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:08.173242 7669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:08.173463 7669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:08.174053 7669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:08.174861 7669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:08.174932 7669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.175009 7669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:08.175062 7669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.183902 7669 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44171
I20260808 15:52:08.183928 7845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44171 every 8 connection(s)
I20260808 15:52:08.184760 7669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:08.186146 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 7669
I20260808 15:52:08.186288 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:08.186640 31025 external_mini_cluster.cc:1428] Setting key 5640211798a1c26e99e09cd8f007c3de
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:08.198027 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.186715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:52165 (local address 127.30.76.126:45997)
0808 15:52:08.186859 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.186864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.187987 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 15:52:08.189228 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.189230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.189276 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.189345 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.190543 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.191058 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.192224 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.192430 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.192511 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.194786 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.194789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.194791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.194814 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.196338 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.196799 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.196801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.196803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.196876 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.197168 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.197170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.197171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.197332 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.197405 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.197821 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.197910 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:52:08.198787 7848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.187166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:52165)
0808 15:52:08.187696 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.187748 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.189042 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.189607 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.189614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.189973 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.190414 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.190427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.191165 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.191167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.192114 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.192124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.192557 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.192559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.192706 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.193232 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.193248 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.194634 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.196437 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.196441 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.196470 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.196714 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.196956 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.196958 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.196959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.197083 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.197417 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.197422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.197624 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.197774 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.197999 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":153,"threads_started":1}
I20260808 15:52:08.199921 7846 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45997
I20260808 15:52:08.200168 7846 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:08.200724 7846 heartbeater.cc:511] Master 127.30.76.126:45997 requested a full tablet report, sending...
I20260808 15:52:08.201670 7188 ts_manager.cc:194] Registered new tserver with Master: ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171)
I20260808 15:52:08.202239 7188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:52165
I20260808 15:52:08.212451 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.76.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.125.keytab.
Entry for principal oryx/127.30.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.125.keytab.
I20260808 15:52:08.241991 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:39677
--webserver_interface=127.30.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.125.keytab
--principal=oryx/127.30.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45997,127.30.76.125:39677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:08.352717 7853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:08.352953 7853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:08.352993 7853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:08.357040 7853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:08.357095 7853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:08.357115 7853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:08.357131 7853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:08.357184 7853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:08.362370 7853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45997,127.30.76.125:39677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.125.keytab
--principal=oryx/127.30.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:39677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
W20260808 15:52:08.362569 7853 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.76.126:45997,127.30.76.125:39677'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 15:52:08.363399 7853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:08.364415 7853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:08.372771 7867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.372781 7864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:08.372852 7853 server_base.cc:1034] running on GCE node
W20260808 15:52:08.372788 7865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:08.373533 7853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:08.374580 7853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:08.375735 7853 hybrid_clock.cc:648] HybridClock initialized: now 1786204328375720 us; error 26 us; skew 500 ppm
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:08.378418 7853 init.cc:377] Logged in from keytab as oryx/127.30.76.125@KRBTEST.COM (short username oryx)
I20260808 15:52:08.379299 7853 webserver.cc:460] Webserver started at http://127.30.76.125:46131/ using document root <none> and password file <none>
I20260808 15:52:08.379889 7853 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.379951 7853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.380195 7853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:08.382141 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/instance:
uuid: "86de954cd47e4e65a093fefa1b027a47"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "5608b0154a176f6a394ebd65aeaf9680"
server_key_iv: "b7974250a60b59e055914007b710b36c"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.382684 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal/instance:
uuid: "86de954cd47e4e65a093fefa1b027a47"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "5608b0154a176f6a394ebd65aeaf9680"
server_key_iv: "b7974250a60b59e055914007b710b36c"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.386312 7853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:52:08.388622 7881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.389778 7853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:08.389900 7853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
uuid: "86de954cd47e4e65a093fefa1b027a47"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "5608b0154a176f6a394ebd65aeaf9680"
server_key_iv: "b7974250a60b59e055914007b710b36c"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.389999 7853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.405143 7853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:08.408047 7853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:08.408247 7853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:08.415036 7853 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.125:39677
I20260808 15:52:08.415318 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.125:39677 every 8 connection(s)
I20260808 15:52:08.416150 7853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
I20260808 15:52:08.417881 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 7853
I20260808 15:52:08.419387 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:08.423702 7958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.125@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:08.432888 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.419421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.30.76.125:39677)
0808 15:52:08.419906 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.419922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.419957 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:52:08.420577 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.420591 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.420909 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.421199 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.422132 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.423131 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.424324 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.424784 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.424902 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.428026 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.428034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.428046 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.428089 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.429841 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.430304 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.430309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.430313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.430479 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.430773 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.430776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.430778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.431277 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.431496 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.431759 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.432052 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":124,"threads_started":1}
I20260808 15:52:08.434727 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.425677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.30.76.126:45997)
0808 15:52:08.425849 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.425854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.426239 (+ 385us) server_negotiation.cc:407] Connection header received
0808 15:52:08.426387 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.426389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.426433 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.426500 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.427296 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.427955 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.428853 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.429199 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.429307 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.431499 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.431503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.431507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.431557 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.432987 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.433448 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.433452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.433454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.433548 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.433816 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.433819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.433821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.434102 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.434187 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.434515 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.434610 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:52:08.434815 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.425592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:41074)
0808 15:52:08.426057 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.426114 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.426273 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.426579 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.426584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.427023 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.427159 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.427165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.428090 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.428093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.428724 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.428732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.429328 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.429331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.429411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.429878 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.429890 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.431368 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.433131 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.433140 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.433144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.433351 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.433653 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.433656 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.433657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.433730 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.434207 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.434214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.434321 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.434463 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.434623 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":180,"threads_started":1}
I20260808 15:52:08.435964 7958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.125@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
I20260808 15:52:08.443863 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.436469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.30.76.125:39677)
0808 15:52:08.436574 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.436578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.436591 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:08.436719 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.436721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.436756 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.436836 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.437197 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.437994 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.438706 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.438932 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.439038 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.440843 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.440846 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.440849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.440876 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.442272 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.442729 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.442732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.442733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.442803 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.443058 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.443060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.443062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.443206 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.443278 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.443646 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.443735 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":39}
I20260808 15:52:08.443866 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.436387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:56102)
0808 15:52:08.436526 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.436540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.436614 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.436846 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.436849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.436996 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.437072 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.437077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.438102 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.438107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.438588 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.438595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.439036 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.439039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.439130 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.439595 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.439607 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.440713 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.442383 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.442386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.442388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.442621 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.442882 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.442885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.442887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.442974 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.443295 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.443298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.443414 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.443556 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.443689 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260808 15:52:08.446956 7958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Bootstrap starting.
I20260808 15:52:08.449595 7958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:08.450435 7958 log.cc:824] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:08.452625 7958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: No bootstrap required, opened a new log
I20260808 15:52:08.456475 7958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } }
I20260808 15:52:08.456678 7958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:08.456714 7958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86de954cd47e4e65a093fefa1b027a47, State: Initialized, Role: FOLLOWER
I20260808 15:52:08.457065 7958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } }
I20260808 15:52:08.457466 7964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } } }
I20260808 15:52:08.457602 7964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:08.458163 7958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 15:52:08.462512 7980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:52:08.462600 7980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:52:08.465154 7853 master_runner.cc:428] Detected that this master 86de954cd47e4e65a093fefa1b027a47 is joining an existing cluster
I20260808 15:52:08.465202 7853 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.76.125:39677
I20260808 15:52:08.476744 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.467107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:56112)
0808 15:52:08.467606 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.467620 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.467728 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.467983 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.467988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.468248 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.468399 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.468406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.469456 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.469460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.470397 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.470409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.472097 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.472101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.472223 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.472769 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.472789 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.473188 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.475155 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.475160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.475414 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.475726 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.475731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.475804 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.476200 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.476203 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.476294 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.476400 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.476519 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":96,"threads_started":1}
I20260808 15:52:08.476850 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.467441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.30.76.125:39677)
0808 15:52:08.467571 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.467577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.467712 (+ 135us) server_negotiation.cc:407] Connection header received
0808 15:52:08.467815 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.467818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.467863 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.467941 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.468516 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.469273 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.471623 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.471945 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.472052 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.473321 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.473325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.473327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.473362 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.475034 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475518 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.475521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.475546 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.475620 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475881 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.475885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.475886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.476099 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.476196 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.476477 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.476588 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39,"spinlock_wait_cycles":9856}
I20260808 15:52:08.477032 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.467137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.30.76.126:45997)
0808 15:52:08.467420 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.467425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.467441 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:08.467611 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.467619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.467667 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.467743 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.468420 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.469274 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.471607 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.471945 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.472052 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.473515 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.473520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.473526 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.473557 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.475286 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475788 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.475791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.475793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.475879 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.476193 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.476197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.476198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.476396 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.476480 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.476808 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.476919 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260808 15:52:08.477764 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.467075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:41090)
0808 15:52:08.467348 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.467364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.467456 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.467917 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.467922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.468173 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.468281 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.468287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.470514 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.470520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.471439 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.471451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.472468 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.472472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.472572 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.472923 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.472937 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.473370 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.475419 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.475423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.475684 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.475984 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.475987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.475989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.476067 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.476535 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.476537 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.476623 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.477482 (+ 859us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.477614 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.126@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
I20260808 15:52:08.491312 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.481781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:56118)
0808 15:52:08.482107 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.482155 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.482270 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.482777 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.482783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.483121 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.483223 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.483229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.483963 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.483967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.484596 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.484605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.485034 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.485038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.485170 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.485670 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.485700 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.487248 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.488990 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.489001 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.489007 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.489313 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.489683 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.489689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.489691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.489797 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.490658 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.490668 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.490809 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.491008 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.491183 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260808 15:52:08.491340 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.482428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.30.76.125:39677)
0808 15:52:08.482545 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.482548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.482558 (+ 10us) server_negotiation.cc:407] Connection header received
0808 15:52:08.482606 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.482607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.482650 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.482740 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.483340 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.483855 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.484718 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.484924 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.485032 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.487379 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.487383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.487387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.487412 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.488874 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.489482 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.489486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.489488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.489577 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.489897 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.489901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.489903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.490142 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.490241 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.491078 (+ 837us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.491193 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:52:08.492869 7188 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.76.125:39677: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "86de954cd47e4e65a093fefa1b027a47"
member_type: NON_VOTER
last_known_addr {
host: "127.30.76.125"
port: 39677
}
attrs {
promote: true
}
}
dest_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f"
cas_config_opid_index: -1
I20260808 15:52:08.493580 7188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } }
I20260808 15:52:08.495872 7917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:08.497543 7917 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Refusing update from remote peer 37ae4ca4d61d4221ab26b3f24e6dd70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 15:52:08.495187 7993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.498302 7992 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
WARNING: no policy specified for oryx/127.30.76.124@KRBTEST.COM; defaulting to no policy
I20260808 15:52:08.499794 7993 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.76.125:39677
I20260808 15:52:08.500267 7992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.500342 7853 master.cc:537] Master@127.30.76.125:39677 shutting down...
I20260808 15:52:08.500371 7992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.124@KRBTEST.COM" created.
I20260808 15:52:08.506529 7917 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.507661 7998 mvcc.cc:205] Tried to move back new op lower bound from 7316292925471924224 to 7316292925308538880. Current Snapshot: MvccSnapshot[applied={T|T < 7316292925347852288}]
I20260808 15:52:08.508384 8002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.508594 8002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.508738 8002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.508764 7964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ae4ca4d61d4221ab26b3f24e6dd70f. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.508885 7964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.508999 8002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.509692 7853 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Raft consensus shutting down.
I20260808 15:52:08.510025 7853 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Raft consensus is shut down!
I20260808 15:52:08.510178 7853 tablet_replica.cc:330] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.124.keytab.
Entry for principal oryx/127.30.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.124.keytab.
I20260808 15:52:08.524461 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:34473
--webserver_interface=127.30.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.124.keytab
--principal=oryx/127.30.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:52:08.526661 7853 master.cc:552] Master@127.30.76.125:39677 shutdown complete.
I20260808 15:52:08.526726 7853 master_runner.cc:305] Clearing existing system tablet
I20260808 15:52:08.526997 7853 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.527061 7853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.528751 7853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.529824 8012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.530090 7853 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.530220 7853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.530254 7853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
uuid: "86de954cd47e4e65a093fefa1b027a47"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "5608b0154a176f6a394ebd65aeaf9680"
server_key_iv: "b7974250a60b59e055914007b710b36c"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.532164 7853 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:08.534447 7853 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 15:52:08.534540 7853 log.cc:1196] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 15:52:08.534843 7853 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Deleting consensus metadata
I20260808 15:52:08.535012 7853 master_runner.cc:315] Copying system tablet from 127.30.76.126:45997
I20260808 15:52:08.536188 7853 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: tablet copy: Beginning tablet copy session from remote peer at address 127.30.76.126:45997
I20260808 15:52:08.545477 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.536785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:41092)
0808 15:52:08.537150 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.537166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.537289 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.537588 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.537591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.537817 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.537939 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.537946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.539039 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.539042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.539997 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.540009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.540617 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.540621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.540727 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.541323 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.541339 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.541766 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.543880 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.543884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.543887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.544146 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.544431 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.544434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.544436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.544530 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.544928 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.544931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.545029 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.545136 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.545274 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":95,"threads_started":1}
I20260808 15:52:08.545483 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.536884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.30.76.126:45997)
0808 15:52:08.537014 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.537019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.537265 (+ 246us) server_negotiation.cc:407] Connection header received
0808 15:52:08.537407 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.537410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.537469 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.537549 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.538070 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.538918 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.540142 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.540492 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.540595 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.541914 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.541918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.541921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.541969 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.543755 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.544246 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.544249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.544251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.544336 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.544621 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.544624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.544626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.544831 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.544924 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.545213 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.545327 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:52:08.546361 7215 tablet_copy_service.cc:140] P 37ae4ca4d61d4221ab26b3f24e6dd70f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 86de954cd47e4e65a093fefa1b027a47 ({username='oryx', principal='oryx/127.30.76.125@KRBTEST.COM'} at 127.0.0.1:41092)
I20260808 15:52:08.546537 7215 tablet_copy_service.cc:161] P 37ae4ca4d61d4221ab26b3f24e6dd70f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 86de954cd47e4e65a093fefa1b027a47 at {username='oryx', principal='oryx/127.30.76.125@KRBTEST.COM'} at 127.0.0.1:41092: session id = 86de954cd47e4e65a093fefa1b027a47-00000000000000000000000000000000
I20260808 15:52:08.548372 7215 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 15:52:08.550034 7853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 15:52:08.552232 7853 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: tablet copy: Starting download of 0 data blocks...
I20260808 15:52:08.552416 7853 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: tablet copy: Starting download of 1 WAL segments...
I20260808 15:52:08.554025 7853 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 15:52:08.555804 7215 tablet_copy_service.cc:342] P 37ae4ca4d61d4221ab26b3f24e6dd70f: Request end of tablet copy session 86de954cd47e4e65a093fefa1b027a47-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.76.125@KRBTEST.COM'} at 127.0.0.1:41092
I20260808 15:52:08.555917 7215 tablet_copy_service.cc:433] P 37ae4ca4d61d4221ab26b3f24e6dd70f: ending tablet copy session 86de954cd47e4e65a093fefa1b027a47-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 86de954cd47e4e65a093fefa1b027a47
I20260808 15:52:08.563963 7853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:08.565603 8023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.565718 8028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.566416 8026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:08.566735 7853 server_base.cc:1034] running on GCE node
I20260808 15:52:08.566888 7853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:08.567178 7853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:08.568322 7853 hybrid_clock.cc:648] HybridClock initialized: now 1786204328568304 us; error 33 us; skew 500 ppm
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:08.570053 7853 init.cc:377] Logged in from keytab as oryx/127.30.76.125@KRBTEST.COM (short username oryx)
I20260808 15:52:08.570503 7853 webserver.cc:460] Webserver started at http://127.30.76.125:42747/ using document root <none> and password file <none>
I20260808 15:52:08.570694 7853 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.570740 7853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.572052 7853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.573068 8042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.573323 7853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.573395 7853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
uuid: "86de954cd47e4e65a093fefa1b027a47"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "5608b0154a176f6a394ebd65aeaf9680"
server_key_iv: "b7974250a60b59e055914007b710b36c"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.573468 7853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.600656 7853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:08.602926 7853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:08.608414 7853 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.125:39677
I20260808 15:52:08.608522 8111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.125:39677 every 8 connection(s)
I20260808 15:52:08.609054 7853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
I20260808 15:52:08.611387 8116 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:52:08.612488 8116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Bootstrap starting.
I20260808 15:52:08.620375 8116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:52:08.620675 8116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: Bootstrap complete.
I20260808 15:52:08.621230 8116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } }
I20260808 15:52:08.621372 8116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 86de954cd47e4e65a093fefa1b027a47, State: Initialized, Role: LEARNER
I20260808 15:52:08.621515 8116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } }
I20260808 15:52:08.621860 8119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } }
I20260808 15:52:08.621979 8116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:08.622053 8119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.624852 8133 catalog_manager.cc:1259] Loaded cluster ID: 2dd02fd64a094281b6e2bfa962b0b00b
I20260808 15:52:08.624970 8133 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: loading cluster ID for follower catalog manager: success
I20260808 15:52:08.627117 8133 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: acquiring CA information for follower catalog manager: success
I20260808 15:52:08.628438 8133 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 15:52:08.665417 8010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:08.665692 8010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:08.665779 8010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:08.669891 8010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:08.669966 8010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:08.670004 8010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:08.670058 8010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:08.670094 8010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:08.675446 8010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.124.keytab
--principal=oryx/127.30.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:34473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:08.676502 8010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:08.677505 8010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:08.685871 8145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.685875 8141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.685896 8139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:08.686090 8010 server_base.cc:1034] running on GCE node
I20260808 15:52:08.687038 8010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:08.688122 8010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:08.689319 8010 hybrid_clock.cc:648] HybridClock initialized: now 1786204328689309 us; error 29 us; skew 500 ppm
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:08.694939 8010 init.cc:377] Logged in from keytab as oryx/127.30.76.124@KRBTEST.COM (short username oryx)
I20260808 15:52:08.695878 8010 webserver.cc:460] Webserver started at http://127.30.76.124:37395/ using document root <none> and password file <none>
I20260808 15:52:08.696426 8010 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.696517 8010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.696749 8010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:08.698724 8010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/instance:
uuid: "ba77ae8813504150b39040df54feaac4"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "24f61246ee7a18056fd69a94d4f7b0b1"
server_key_iv: "df756d180c869cdf34e38c76ef6e2e03"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.699295 8010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal/instance:
uuid: "ba77ae8813504150b39040df54feaac4"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "24f61246ee7a18056fd69a94d4f7b0b1"
server_key_iv: "df756d180c869cdf34e38c76ef6e2e03"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.703055 8010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 15:52:08.705471 8156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.706676 8010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:08.706825 8010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
uuid: "ba77ae8813504150b39040df54feaac4"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "24f61246ee7a18056fd69a94d4f7b0b1"
server_key_iv: "df756d180c869cdf34e38c76ef6e2e03"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.706934 8010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.724291 8010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:08.728070 8010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:08.728284 8010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:08.735783 8010 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.124:34473
I20260808 15:52:08.736074 8231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.124:34473 every 8 connection(s)
I20260808 15:52:08.736864 8010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
I20260808 15:52:08.739565 8232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:08.742758 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 8010
I20260808 15:52:08.743396 8232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:52:08.747707 7474 heartbeater.cc:503] Master 127.30.76.126:45997 was elected leader, sending a full tablet report...
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.124@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.76.124@KRBTEST.COM
I20260808 15:52:08.755915 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.745479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.30.76.126:45997)
0808 15:52:08.745610 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.745615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.745816 (+ 201us) server_negotiation.cc:407] Connection header received
0808 15:52:08.746201 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.746204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.746274 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.746361 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.747150 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.747794 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.749166 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.749381 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.749495 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.752022 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.752030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.752033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.752056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.753669 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.754321 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.754323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.754324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.754397 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.754752 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.754757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.754758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.754942 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.755052 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.755615 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.755743 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:52:08.756872 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.745285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:41098)
0808 15:52:08.745689 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.745726 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.746004 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.746413 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.746418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.746765 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.747000 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.747013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.747977 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.747981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.749041 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.749050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.749529 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.749533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.749701 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.750265 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.750281 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.751861 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.753778 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.753785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.753797 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.754223 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.754498 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.754502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.754504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.754652 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.755052 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.755061 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.755376 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.755582 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.755882 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":149,"threads_started":1}
I20260808 15:52:08.756876 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.744500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.30.76.124:34473)
0808 15:52:08.744988 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.745010 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.745050 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:52:08.745916 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.745939 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.746166 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.746352 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.747201 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.748319 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.749293 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.749660 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.749779 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.752432 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.752438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.752450 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.752479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.754261 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.754729 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.754735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.754736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.754818 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.755094 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.755098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.755100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.755391 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.755623 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.755839 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.755978 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":147,"threads_started":1}
I20260808 15:52:08.758086 8232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.124@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
I20260808 15:52:08.765702 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.758561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:56134)
0808 15:52:08.758652 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.758667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.758741 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.759177 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.759180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.759308 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.759395 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.759400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.760322 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.760324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.760843 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.760851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.761264 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.761266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.761339 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.761784 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.761804 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.762910 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.764554 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.764557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.764558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.764741 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.764970 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.764972 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.764973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.765042 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.765354 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.765356 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.765426 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.765503 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.765595 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 15:52:08.765766 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.758686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56134 (local address 127.30.76.125:39677)
0808 15:52:08.758933 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.758937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.758949 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:52:08.758986 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.758988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.759025 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.759154 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.759497 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.760223 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.760952 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.761171 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.761247 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.763042 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.763045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.763047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.763069 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.764460 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.764821 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.764824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.764825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.764894 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.765113 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.765115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.765116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.765277 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.765350 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.765560 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.765650 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260808 15:52:08.766572 8232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.124@KRBTEST.COM for oryx/127.30.76.124@KRBTEST.COM
I20260808 15:52:08.773550 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.766976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.0.0.1:48734)
0808 15:52:08.767085 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.767099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.767172 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.767345 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.767347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.767464 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.767571 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.767581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.768273 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.768276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.768742 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.768750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.769290 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.769293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.769386 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.769822 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.769835 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.770954 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.772399 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.772402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.772403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.772580 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.772808 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.772810 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.772811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.772880 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.773181 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.773183 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.773250 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.773325 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.773407 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 15:52:08.773653 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.767063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.30.76.124:34473)
0808 15:52:08.767166 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.767169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.767183 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:52:08.767221 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.767223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.767254 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.767319 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.767678 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.768140 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.768886 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.769130 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.769233 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.771072 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.771075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.771076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.771096 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.772306 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.772662 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.772664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.772665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.772731 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.772953 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.772955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.772956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.773095 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.773176 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.773382 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.773463 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:52:08.777034 8232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Bootstrap starting.
I20260808 15:52:08.779331 8232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:08.780082 8232 log.cc:824] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:08.781880 8232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: No bootstrap required, opened a new log
I20260808 15:52:08.784842 8232 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } }
I20260808 15:52:08.785041 8232 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:08.785135 8232 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba77ae8813504150b39040df54feaac4, State: Initialized, Role: FOLLOWER
I20260808 15:52:08.785595 8232 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } }
I20260808 15:52:08.786069 8239 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } } }
I20260808 15:52:08.786182 8239 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:08.786764 8232 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 15:52:08.791497 8262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:52:08.791602 8262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:52:08.794957 8010 master_runner.cc:428] Detected that this master ba77ae8813504150b39040df54feaac4 is joining an existing cluster
I20260808 15:52:08.795006 8010 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.76.124:34473
I20260808 15:52:08.806744 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.797406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.30.76.126:45997)
0808 15:52:08.797543 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.797548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.797714 (+ 166us) server_negotiation.cc:407] Connection header received
0808 15:52:08.797858 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.797862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.797939 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.798022 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.798737 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.799588 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.801772 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.802102 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.802179 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.803455 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.803458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.803460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.803484 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.804946 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.805600 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.805603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.805604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.805684 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.805954 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.805956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.805957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.806107 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.806219 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.806489 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.806602 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:08.806764 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.796979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:41112)
0808 15:52:08.797615 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.797636 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.797755 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.798146 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.798149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.798440 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.798584 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.798591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.799739 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.799742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.801632 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.801644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.802332 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.802337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.802431 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.802827 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.802842 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.803310 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.805048 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.805051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.805054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.805515 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.805776 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.805779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.805781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.805874 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.806213 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.806216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.806317 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.806430 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.806565 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":131,"thread_start_us":111,"threads_started":1}
I20260808 15:52:08.810256 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 7274
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204327, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.126@KRBTEST.COM for oryx/127.30.76.124@KRBTEST.COM
I20260808 15:52:08.825026 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.797129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.0.0.1:48738)
0808 15:52:08.797867 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.797880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.797989 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.798300 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.798306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.798647 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.798775 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.798783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.799789 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.799798 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.800790 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.800800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.801481 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.801484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.801714 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.802349 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.802361 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.819859 (+ 17498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.823414 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.823417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.823418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.823696 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.823993 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.823999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.824002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.824080 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.824472 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.824474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.824567 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.824688 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.824827 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":173,"spinlock_wait_cycles":7424,"thread_start_us":100,"threads_started":1}
I20260808 15:52:08.825027 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.797112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56144 (local address 127.30.76.125:39677)
0808 15:52:08.797354 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.797360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.798052 (+ 692us) server_negotiation.cc:407] Connection header received
0808 15:52:08.798221 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.798223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.798275 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.798325 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.798967 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.799749 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.800939 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.801294 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.801358 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.821295 (+ 19937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.821301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.821307 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.821353 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.823040 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.823646 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.823653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.823657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.823745 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.824058 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.824061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.824063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.824302 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.824376 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.824769 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.824896 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260808 15:52:08.825521 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.797575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:56144)
0808 15:52:08.797977 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.797989 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.798102 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.798489 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.798495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.798744 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.798845 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.798853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.799890 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.799893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.800792 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.800800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.801490 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.801492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.801718 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.820576 (+ 18858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.820597 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.821083 (+ 486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.823192 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.823195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.823197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.823503 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.823859 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.823862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.823864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.823956 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.824449 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.824454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.824587 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.825182 (+ 595us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.825324 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":121,"spinlock_wait_cycles":51840,"thread_start_us":103,"threads_started":1}
I20260808 15:52:08.825534 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:46523
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:45997
--webserver_port=45877
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:52:08.825505 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.797229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.30.76.124:34473)
0808 15:52:08.797464 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.797468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.797940 (+ 472us) server_negotiation.cc:407] Connection header received
0808 15:52:08.798082 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.798086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.798121 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.798173 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.798896 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.799667 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.800915 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.801244 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.801318 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.819979 (+ 18661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.819985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.819989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.820019 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.822657 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.823800 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.823805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.823809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.823897 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.824176 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.824179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.824181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.824359 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.824443 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.825280 (+ 837us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.825368 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"spinlock_wait_cycles":5504}
I20260808 15:52:08.829537 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.810285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.0.0.1:48754)
0808 15:52:08.810403 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.810422 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.810507 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.817639 (+ 7132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.817643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.817860 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.817980 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.817986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.818764 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.818768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.819307 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.819315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.820307 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.820314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.820441 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.822825 (+ 2384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.822839 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.824488 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.827792 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.827798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.827801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.828064 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.828437 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.828444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.828446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.828544 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.829008 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.829013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.829114 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.829230 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.829381 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 15:52:08.829553 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.810348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.30.76.124:34473)
0808 15:52:08.817390 (+ 7042us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.817396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.817408 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:52:08.817467 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.817469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.817509 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.817620 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.818117 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.818654 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.819428 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.819827 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.825721 (+ 5894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.825827 (+ 106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.825829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.825831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.825843 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.827655 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.828203 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.828208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.828211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.828315 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.828669 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.828676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.828677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.828915 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.829002 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.829282 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.829389 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6979,"thread_start_us":98,"threads_started":1}
I20260808 15:52:08.830367 7188 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.76.124:34473: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ba77ae8813504150b39040df54feaac4"
member_type: NON_VOTER
last_known_addr {
host: "127.30.76.124"
port: 34473
}
attrs {
promote: true
}
}
dest_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f"
cas_config_opid_index: 5
I20260808 15:52:08.830936 7188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } }
I20260808 15:52:08.832091 7992 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ba77ae8813504150b39040df54feaac4 (127.30.76.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.833035 8192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:08.834008 7992 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.76.124:34473
I20260808 15:52:08.834237 7992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.834384 7992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:08.834511 8192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Refusing update from remote peer 37ae4ca4d61d4221ab26b3f24e6dd70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 15:52:08.834878 8010 master.cc:537] Master@127.30.76.124:34473 shutting down...
I20260808 15:52:08.835052 7993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:52:08.836309 7173 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 15:52:08.837230 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.832504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.30.76.125:39677)
0808 15:52:08.832645 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.832651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.832799 (+ 148us) server_negotiation.cc:407] Connection header received
0808 15:52:08.832974 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.832977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.833048 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.833139 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:08.833572 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.834320 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.835778 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.836389 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.836466 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.836657 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.837024 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.837100 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:52:08.837622 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.832274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:56148)
0808 15:52:08.832734 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.832748 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.832863 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.833313 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.833316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.833335 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:08.833429 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.833436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.834466 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.834469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.835625 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.835643 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.836757 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.836760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.836853 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.836899 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.837414 (+ 515us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.837508 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392}
W20260808 15:52:08.837898 7173 consensus_peers.cc:487] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f -> Peer ba77ae8813504150b39040df54feaac4 (127.30.76.124:34473): Couldn't send request to peer ba77ae8813504150b39040df54feaac4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 15:52:08.838127 8075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Refusing update from remote peer 37ae4ca4d61d4221ab26b3f24e6dd70f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 15:52:08.838591 7994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 15:52:08.839754 8075 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ba77ae8813504150b39040df54feaac4 (127.30.76.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.841054 8278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.841253 8278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.841327 8119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ae4ca4d61d4221ab26b3f24e6dd70f. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: NON_VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: true } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.841420 8119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.841465 8010 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:08.841756 8010 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:08.841873 8010 tablet_replica.cc:330] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: stopping tablet replica
I20260808 15:52:08.842293 7994 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: attempting to promote NON_VOTER 86de954cd47e4e65a093fefa1b027a47 to VOTER
I20260808 15:52:08.842756 7994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } }
I20260808 15:52:08.844429 8075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 LEARNER]: Refusing update from remote peer 37ae4ca4d61d4221ab26b3f24e6dd70f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 15:52:08.845345 7993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 15:52:08.846560 7173 consensus_peers.cc:487] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f -> Peer ba77ae8813504150b39040df54feaac4 (127.30.76.124:34473): Couldn't send request to peer ba77ae8813504150b39040df54feaac4. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260808 15:52:08.847342 7992 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.847865 8075 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.848865 7992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.848971 7992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:08.849128 7992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.849215 7992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:08.849543 8119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.849654 8119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:08.857945 8010 master.cc:552] Master@127.30.76.124:34473 shutdown complete.
I20260808 15:52:08.858063 8010 master_runner.cc:305] Clearing existing system tablet
I20260808 15:52:08.858438 8010 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.858541 8010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.859774 8010 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.860630 8281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.860869 8010 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.861037 8010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.861075 8010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
uuid: "ba77ae8813504150b39040df54feaac4"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "24f61246ee7a18056fd69a94d4f7b0b1"
server_key_iv: "df756d180c869cdf34e38c76ef6e2e03"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.862787 8010 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:08.864710 8010 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 15:52:08.864818 8010 log.cc:1196] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 15:52:08.865044 8010 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Deleting consensus metadata
I20260808 15:52:08.865234 8010 master_runner.cc:315] Copying system tablet from 127.30.76.126:45997
I20260808 15:52:08.866362 8010 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: tablet copy: Beginning tablet copy session from remote peer at address 127.30.76.126:45997
I20260808 15:52:08.875033 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.866979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.30.76.126:45997)
0808 15:52:08.867107 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:08.867112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:08.867352 (+ 240us) server_negotiation.cc:407] Connection header received
0808 15:52:08.867481 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:08.867483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:08.867553 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:08.867623 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:08.868066 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.868747 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.869948 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.870235 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.870348 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.871558 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:08.871562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:08.871565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:08.871590 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:08.873191 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.873678 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.873682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.873687 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.873780 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:08.874131 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:08.874134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:08.874136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:08.874316 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:08.874486 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:08.874740 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 15:52:08.874862 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:08.875052 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:08.866929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:41118)
0808 15:52:08.867243 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:08.867255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:08.867398 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:08.867637 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:08.867640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:08.867847 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:08.867935 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.867941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.868859 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.868864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.869824 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:08.869834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:08.870363 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:08.870366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:08.870470 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:08.871044 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:08.871065 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:08.871394 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:08.873301 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.873306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.873308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.873575 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.873876 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:08.873879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:08.873882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:08.873972 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:08.874444 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:08.874450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:08.874556 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 15:52:08.874684 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 15:52:08.874873 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":91,"threads_started":1}
I20260808 15:52:08.875838 7215 tablet_copy_service.cc:140] P 37ae4ca4d61d4221ab26b3f24e6dd70f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ba77ae8813504150b39040df54feaac4 ({username='oryx', principal='oryx/127.30.76.124@KRBTEST.COM'} at 127.0.0.1:41118)
I20260808 15:52:08.875958 7215 tablet_copy_service.cc:161] P 37ae4ca4d61d4221ab26b3f24e6dd70f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ba77ae8813504150b39040df54feaac4 at {username='oryx', principal='oryx/127.30.76.124@KRBTEST.COM'} at 127.0.0.1:41118: session id = ba77ae8813504150b39040df54feaac4-00000000000000000000000000000000
I20260808 15:52:08.877137 7215 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 15:52:08.878239 8010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 15:52:08.880024 8010 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: tablet copy: Starting download of 0 data blocks...
I20260808 15:52:08.880227 8010 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: tablet copy: Starting download of 1 WAL segments...
I20260808 15:52:08.881630 8010 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 15:52:08.883376 7215 tablet_copy_service.cc:342] P 37ae4ca4d61d4221ab26b3f24e6dd70f: Request end of tablet copy session ba77ae8813504150b39040df54feaac4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.76.124@KRBTEST.COM'} at 127.0.0.1:41118
I20260808 15:52:08.883473 7215 tablet_copy_service.cc:433] P 37ae4ca4d61d4221ab26b3f24e6dd70f: ending tablet copy session ba77ae8813504150b39040df54feaac4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ba77ae8813504150b39040df54feaac4
I20260808 15:52:08.889869 8010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:08.891634 8289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.891857 8296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:08.891965 8292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:08.892095 8010 server_base.cc:1034] running on GCE node
I20260808 15:52:08.892423 8010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:08.892752 8010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:08.893893 8010 hybrid_clock.cc:648] HybridClock initialized: now 1786204328893876 us; error 33 us; skew 500 ppm
Aug 08 15:52:08 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204328, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:08.895589 8010 init.cc:377] Logged in from keytab as oryx/127.30.76.124@KRBTEST.COM (short username oryx)
I20260808 15:52:08.896159 8010 webserver.cc:460] Webserver started at http://127.30.76.124:36697/ using document root <none> and password file <none>
I20260808 15:52:08.896385 8010 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:08.896440 8010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:08.897535 8010 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.898487 8306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:08.898720 8010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:08.898819 8010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
uuid: "ba77ae8813504150b39040df54feaac4"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "24f61246ee7a18056fd69a94d4f7b0b1"
server_key_iv: "df756d180c869cdf34e38c76ef6e2e03"
server_key_version: "encryptionkey@0"
I20260808 15:52:08.898926 8010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:08.909586 8010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:08.911779 8010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:08.916734 8010 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.124:34473
I20260808 15:52:08.916851 8377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.124:34473 every 8 connection(s)
I20260808 15:52:08.917376 8010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
I20260808 15:52:08.918840 8378 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:52:08.919781 8378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Bootstrap starting.
I20260808 15:52:08.928757 8378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:52:08.929065 8378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Bootstrap complete.
I20260808 15:52:08.929539 8378 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } }
I20260808 15:52:08.929683 8378 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba77ae8813504150b39040df54feaac4, State: Initialized, Role: LEARNER
I20260808 15:52:08.929803 8378 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } }
I20260808 15:52:08.930138 8381 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:08.930310 8381 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:52:08.930312 8378 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:08.933795 8402 catalog_manager.cc:1259] Loaded cluster ID: 2dd02fd64a094281b6e2bfa962b0b00b
I20260808 15:52:08.933895 8402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: loading cluster ID for follower catalog manager: success
I20260808 15:52:08.935429 8402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: acquiring CA information for follower catalog manager: success
I20260808 15:52:08.936913 8402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:52:08.981444 7663 heartbeater.cc:503] Master 127.30.76.126:45997 was elected leader, sending a full tablet report...
W20260808 15:52:08.995361 8275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:08.995715 8275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:08.995787 8275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:09.002027 8275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:09.002112 8275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:09.002204 8275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:09.010351 8275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:46523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=45877
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:09.011961 8275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:09.013387 8275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:09.025309 8412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:09.025338 8415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:09.025305 8413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:09.026041 8275 server_base.cc:1034] running on GCE node
I20260808 15:52:09.026435 8275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:09.027047 8275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:09.028203 8275 hybrid_clock.cc:648] HybridClock initialized: now 1786204329028162 us; error 49 us; skew 500 ppm
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:09.031096 8275 init.cc:377] Logged in from keytab as oryx/127.30.76.65@KRBTEST.COM (short username oryx)
I20260808 15:52:09.032097 8275 webserver.cc:460] Webserver started at http://127.30.76.65:45877/ using document root <none> and password file <none>
I20260808 15:52:09.032698 8275 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:09.032799 8275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:09.036900 8275 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:09.039134 8422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.040484 8275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:09.040632 8275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "9ef601be8bcf41b887bff3d1e29f1af4"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "996bdd9aee17d6503810a6bd0f0d3d67"
server_key_iv: "cb654fa231625cd2c5ba7beb8aa8ecf9"
server_key_version: "encryptionkey@0"
I20260808 15:52:09.041216 8275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:09.058816 8275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:09.061643 8275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:09.061846 8275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:09.062433 8275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:09.063412 8275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:09.063516 8275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.063612 8275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:09.063661 8275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.073312 8275 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:46523
I20260808 15:52:09.073334 8574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:46523 every 8 connection(s)
I20260808 15:52:09.074409 8275 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:09.078128 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 8275
I20260808 15:52:09.078347 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 7482
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:09.091738 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.077847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:38457 (local address 127.30.76.126:45997)
0808 15:52:09.077975 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.077981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.078078 (+ 97us) server_negotiation.cc:407] Connection header received
0808 15:52:09.078600 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.078606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.078668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.078721 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.081027 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.081759 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.083334 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.083692 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.083779 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.087257 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.087264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.087269 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.087301 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.089384 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.089983 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.089989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.089994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.090076 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.090453 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.090458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.090463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.090687 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.090757 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.091441 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.091587 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:52:09.093089 8582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.077706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:38457)
0808 15:52:09.077975 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.077990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.078374 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.079104 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.079114 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.079753 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.080360 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.080378 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.081926 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.081931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.083187 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.083200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.083986 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.083992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.084203 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.085241 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.085265 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.087070 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.089511 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.089519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.089536 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.089863 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.090183 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.090188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.090193 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.090356 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.090834 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.090843 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.091157 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.091888 (+ 731us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.092197 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":93,"threads_started":1}
I20260808 15:52:09.096750 8575 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45997
I20260808 15:52:09.097826 8575 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.098470 8575 heartbeater.cc:511] Master 127.30.76.126:45997 requested a full tablet report, sending...
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.124@KRBTEST.COM
I20260808 15:52:09.102084 7188 ts_manager.cc:194] Re-registered known tserver with Master: 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
I20260808 15:52:09.102981 7188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:38457
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
I20260808 15:52:09.105535 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:37185
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:45997
--webserver_port=39591
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:52:09.106650 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.076675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.30.76.65:51961)
0808 15:52:09.077379 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.077393 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.078370 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.079108 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.079116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.080687 (+ 1571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.080852 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.080862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.098011 (+ 17149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.098014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.098822 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.098833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.099646 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.099653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.099746 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.100322 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.100336 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.102009 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.104807 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.104813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.104818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.105099 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.105433 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.105439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.105444 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.105540 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.106015 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.106021 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.106122 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.106262 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.106400 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":22,"spinlock_wait_cycles":29056,"thread_start_us":143,"threads_started":1}
I20260808 15:52:09.107095 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.076302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:51961 (local address 127.30.76.124:34473)
0808 15:52:09.076581 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.076586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.077474 (+ 888us) server_negotiation.cc:407] Connection header received
0808 15:52:09.078612 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.078618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.078695 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.078761 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.082205 (+ 3444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.083700 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.099005 (+ 15305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.099479 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.099643 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.102470 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.102481 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.102487 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.102524 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.104677 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.105217 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.105221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.105222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.105317 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.105643 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.105648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.105651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.105843 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.105953 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.106319 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.106451 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260808 15:52:09.107782 8576 heartbeater.cc:347] Connected to a master server at 127.30.76.124:34473
I20260808 15:52:09.107889 8576 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.108129 8576 heartbeater.cc:511] Master 127.30.76.124:34473 requested a full tablet report, sending...
I20260808 15:52:09.109659 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.076330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:52553 (local address 127.30.76.125:39677)
0808 15:52:09.076458 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.076462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.077320 (+ 858us) server_negotiation.cc:407] Connection header received
0808 15:52:09.096904 (+ 19584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.096910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.096980 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.097033 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.098639 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.099319 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.101262 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.101584 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.101665 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.105544 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.105551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.105556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.105587 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.107791 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.108321 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.108327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.108331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.108438 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.108751 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.108756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.108759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.109007 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.109117 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.109441 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.109545 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:52:09.109637 8329 ts_manager.cc:194] Registered new tserver with Master: 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
I20260808 15:52:09.109712 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.076675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.30.76.65:52553)
0808 15:52:09.077170 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.077221 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.078373 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.097180 (+ 18807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.097185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.097403 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.097546 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.097556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.099465 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.099467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.100460 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.100472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.101939 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.101943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.102032 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.102494 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.102509 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.105388 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.107924 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.107927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.107929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.108166 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.108550 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.108552 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.108553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.108631 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.109122 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.109125 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.109215 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.109319 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.109514 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"spinlock_wait_cycles":6016,"thread_start_us":136,"threads_started":1}
I20260808 15:52:09.110627 8577 heartbeater.cc:347] Connected to a master server at 127.30.76.125:39677
I20260808 15:52:09.110749 8577 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.110898 8577 heartbeater.cc:511] Master 127.30.76.125:39677 requested a full tablet report, sending...
I20260808 15:52:09.112172 8065 ts_manager.cc:194] Registered new tserver with Master: 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
I20260808 15:52:09.204172 7846 heartbeater.cc:503] Master 127.30.76.126:45997 was elected leader, sending a full tablet report...
W20260808 15:52:09.225785 8583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:09.226101 8583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:09.226197 8583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:09.230911 8583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:09.231005 8583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:09.231107 8583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:09.237241 8583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:37185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=39591
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:09.238482 8583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:09.239650 8583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:09.250001 8588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:09.250072 8583 server_base.cc:1034] running on GCE node
W20260808 15:52:09.250012 8596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:09.250027 8593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:09.250751 8583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:09.251369 8583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:09.252564 8583 hybrid_clock.cc:648] HybridClock initialized: now 1786204329252556 us; error 47 us; skew 500 ppm
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:09.255232 8583 init.cc:377] Logged in from keytab as oryx/127.30.76.66@KRBTEST.COM (short username oryx)
I20260808 15:52:09.256273 8583 webserver.cc:460] Webserver started at http://127.30.76.66:39591/ using document root <none> and password file <none>
I20260808 15:52:09.256939 8583 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:09.257017 8583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:09.260643 8583 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 15:52:09.262624 8604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.263876 8583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:09.263969 8583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "413997d0e55e42aa8fc81c653ddd8842"
format_stamp: "Formatted at 2026-08-08 15:52:07 on dist-test-slave-sb2z"
server_key: "383cac8e65fd76785e6f8c7a717506ff"
server_key_iv: "cf27b9cb2b347509ed7c337c85463dac"
server_key_version: "encryptionkey@0"
I20260808 15:52:09.264467 8583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:09.278172 8583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:09.281209 8583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:09.281379 8583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:09.281944 8583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:09.282886 8583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:09.282946 8583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.282992 8583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:09.283016 8583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.292529 8583 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:37185
I20260808 15:52:09.292550 8761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:37185 every 8 connection(s)
I20260808 15:52:09.293632 8583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:09.294723 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 8583
I20260808 15:52:09.294898 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 7669
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.124@KRBTEST.COM
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:09.313663 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:44171
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:45997
--webserver_port=38929
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:52:09.315348 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.297517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:50433 (local address 127.30.76.125:39677)
0808 15:52:09.298208 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.298215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.299745 (+ 1530us) server_negotiation.cc:407] Connection header received
0808 15:52:09.301241 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.301248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.301314 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.301373 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.303218 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.304092 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.306020 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.306402 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.306491 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.310304 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.310309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.310312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.310343 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.312404 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.313181 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.313188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.313193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.313297 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.313799 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.313803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.313805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.314076 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.314172 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.314999 (+ 827us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.315148 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571}
I20260808 15:52:09.315620 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.296339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:42215 (local address 127.30.76.124:34473)
0808 15:52:09.296538 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.296547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.300189 (+ 3642us) server_negotiation.cc:407] Connection header received
0808 15:52:09.304508 (+ 4319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.304514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.304584 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.304646 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.305378 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.306247 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.307653 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.308073 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.308172 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.311395 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.311400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.311402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.311464 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.313532 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.314083 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.314087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.314089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.314199 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.314548 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.314552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.314554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.314791 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.314877 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.315272 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.315426 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:52:09.317389 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.296078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.30.76.66:50433)
0808 15:52:09.299467 (+ 3389us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.299603 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.300989 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.301786 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.301797 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.302333 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.303004 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.303024 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.304281 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.304285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.305851 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.305863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.306711 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.306715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.307007 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.307811 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.307834 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.310070 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.312575 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.312585 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.312604 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.313022 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.313460 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.313463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.313466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.313672 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.314272 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.314283 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.314664 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.315646 (+ 982us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.316191 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3158,"mutex_wait_us":4,"thread_start_us":1063,"threads_started":1}
I20260808 15:52:09.317476 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.304393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:33303 (local address 127.30.76.126:45997)
0808 15:52:09.304567 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.304574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.306421 (+ 1847us) server_negotiation.cc:407] Connection header received
0808 15:52:09.306612 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.306618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.306693 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.306751 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.307951 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.308857 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.310195 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.310600 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.310703 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.313519 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.313524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.313531 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.313565 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.315398 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.315962 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.315966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.315968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.316074 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.316443 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.316446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.316448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.316690 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.316792 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.317166 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.317300 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:52:09.317987 8770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.303688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:33303)
0808 15:52:09.306311 (+ 2623us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.306328 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.306497 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.306963 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.306969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.307455 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.307724 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.307738 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.309012 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.309018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.310052 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.310067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.310904 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.310910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.311033 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.311752 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.311772 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.313355 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.315568 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.315571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.315573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.315861 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.316197 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.316200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.316205 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.316308 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.316844 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.316847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.316953 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.317627 (+ 674us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.317775 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2534,"thread_start_us":379,"threads_started":1}
I20260808 15:52:09.319180 8766 heartbeater.cc:347] Connected to a master server at 127.30.76.125:39677
I20260808 15:52:09.319445 8764 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45997
I20260808 15:52:09.319608 8764 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.319608 8766 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.320279 8764 heartbeater.cc:511] Master 127.30.76.126:45997 requested a full tablet report, sending...
I20260808 15:52:09.320279 8766 heartbeater.cc:511] Master 127.30.76.125:39677 requested a full tablet report, sending...
I20260808 15:52:09.321593 7188 ts_manager.cc:194] Re-registered known tserver with Master: 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185)
I20260808 15:52:09.322458 7188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:33303
I20260808 15:52:09.322467 8065 ts_manager.cc:194] Registered new tserver with Master: 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185)
I20260808 15:52:09.324337 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.296079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.30.76.66:42215)
0808 15:52:09.300091 (+ 4012us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.300107 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.304320 (+ 4213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.304806 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.304809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.305064 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.305216 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.305224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.306395 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.306399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.307439 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.307453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.308347 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.308351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.308486 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.309110 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.309129 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.311249 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.313658 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.313663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.313665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.313953 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.314323 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.314326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.314328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.314419 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.314973 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.314976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.315065 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.315766 (+ 701us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.316476 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3924,"mutex_wait_us":2934,"thread_start_us":150,"threads_started":1}
I20260808 15:52:09.325433 8765 heartbeater.cc:347] Connected to a master server at 127.30.76.124:34473
I20260808 15:52:09.325538 8765 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.325811 8765 heartbeater.cc:511] Master 127.30.76.124:34473 requested a full tablet report, sending...
I20260808 15:52:09.326656 8329 ts_manager.cc:194] Registered new tserver with Master: 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185)
I20260808 15:52:09.418758 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.414734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.0.0.1:48770)
0808 15:52:09.414969 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.414983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.415088 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.415407 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.415409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.415424 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:09.415572 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.415600 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.416703 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.416706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.417473 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.417482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.418415 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.418417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.418489 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.418512 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.418561 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.418626 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
I20260808 15:52:09.418829 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.414822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.30.76.124:34473)
0808 15:52:09.414961 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.414966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.415072 (+ 106us) server_negotiation.cc:407] Connection header received
0808 15:52:09.415189 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.415192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.415264 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.415378 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:09.415734 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.416581 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.417609 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.418275 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.418412 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.418586 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.418626 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.418702 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:52:09.419597 8339 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 15:52:09.420022 8381 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ae4ca4d61d4221ab26b3f24e6dd70f. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:09.420194 8381 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: This master's current role is: LEARNER
W20260808 15:52:09.438148 8771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:09.438400 8771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:09.438444 8771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:09.442727 8771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:09.442800 8771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:09.442881 8771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:09.450591 8771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:44171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=38929
--tserver_master_addrs=127.30.76.126:45997,127.30.76.125:39677,127.30.76.124:34473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:09.451884 8771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:09.453121 8771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:09.463094 8779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:09.463145 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:09.463140 8781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:09.463299 8771 server_base.cc:1034] running on GCE node
I20260808 15:52:09.463797 8771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:09.464427 8771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:09.465629 8771 hybrid_clock.cc:648] HybridClock initialized: now 1786204329465617 us; error 30 us; skew 500 ppm
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:09.468708 8771 init.cc:377] Logged in from keytab as oryx/127.30.76.67@KRBTEST.COM (short username oryx)
I20260808 15:52:09.469599 8771 webserver.cc:460] Webserver started at http://127.30.76.67:38929/ using document root <none> and password file <none>
I20260808 15:52:09.470103 8771 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:09.470175 8771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:09.474136 8771 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:09.476179 8795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.477494 8771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:09.477595 8771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "ea97b41c75344804ae9678ca333e2d65"
format_stamp: "Formatted at 2026-08-08 15:52:08 on dist-test-slave-sb2z"
server_key: "7c6a0b3db28be844b3cab6f2da2de9f4"
server_key_iv: "090ff1b68f0820d16eb66080c0b38c7e"
server_key_version: "encryptionkey@0"
I20260808 15:52:09.478094 8771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:09.503942 8771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:09.507635 8771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:09.507849 8771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:09.508404 8771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:09.509234 8771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:09.509284 8771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.509325 8771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:09.509346 8771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:09.518244 8771 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44171
I20260808 15:52:09.518332 8947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44171 every 8 connection(s)
I20260808 15:52:09.519294 8771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:09.523664 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 8771
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.124@KRBTEST.COM
Aug 08 15:52:09 dist-test-slave-sb2z krb5kdc[7114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204329, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.125@KRBTEST.COM
I20260808 15:52:09.537256 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.521507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:39303 (local address 127.30.76.125:39677)
0808 15:52:09.521623 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.521627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.522997 (+ 1370us) server_negotiation.cc:407] Connection header received
0808 15:52:09.523490 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.523493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.523569 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.523626 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.527756 (+ 4130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.528493 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.530130 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.530481 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.530565 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.533264 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.533269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.533272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.533304 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.534967 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535446 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.535452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.535455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.535573 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535902 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.535907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.535909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.536095 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.536181 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.536958 (+ 777us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.537074 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:52:09.537941 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.521159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:53823 (local address 127.30.76.126:45997)
0808 15:52:09.521295 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.521300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.523186 (+ 1886us) server_negotiation.cc:407] Connection header received
0808 15:52:09.523506 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.523512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.523602 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.523674 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.525741 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.526488 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528256 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.528596 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528690 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.533177 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.533182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.533187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.533218 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.535588 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.536077 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.536080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.536082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.536178 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.536501 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.536505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.536507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.536752 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.536855 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.537667 (+ 812us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.537781 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":57}
I20260808 15:52:09.538277 8955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.521621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.30.76.67:59335)
0808 15:52:09.522251 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.522301 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.523255 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.524036 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.524044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.524709 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.525516 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.525537 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.526689 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.526693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.528117 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.528135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.528844 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.529109 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.529949 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.529965 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.532566 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.534811 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.534819 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.534838 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.535198 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.535515 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.535520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.535731 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.536230 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.536238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.536651 (+ 413us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.536878 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.537198 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":266,"spinlock_wait_cycles":17792,"thread_start_us":734,"threads_started":1}
I20260808 15:52:09.538545 8961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.521621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.30.76.67:39303)
0808 15:52:09.522896 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.522907 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.523255 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.524036 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.524044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.524724 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.527602 (+ 2878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.527613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.528637 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.529980 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.529992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.530782 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.530787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.530878 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.531643 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.531658 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.533095 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.535090 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.535095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.535334 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.535695 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.535699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.535787 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.536252 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.536254 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.536651 (+ 397us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.537392 (+ 741us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.537519 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"mutex_wait_us":2306,"spinlock_wait_cycles":38784,"thread_start_us":177,"threads_started":1}
I20260808 15:52:09.539291 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.521831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:59335 (local address 127.30.76.124:34473)
0808 15:52:09.522709 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.522713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.522728 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:09.523490 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.523493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.523569 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.523634 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.525697 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.526474 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528254 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.528597 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528689 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.532799 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.532802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.532805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.532829 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.534667 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535317 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.535320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.535322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.535410 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535850 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.535856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.535860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.536072 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.536171 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.539012 (+ 2841us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.539141 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772}
I20260808 15:52:09.540117 8952 heartbeater.cc:347] Connected to a master server at 127.30.76.125:39677
I20260808 15:52:09.540356 8952 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.540489 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.521621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:53823)
0808 15:52:09.523099 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.523109 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.523263 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.524039 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.524047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.524731 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:09.525516 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.525538 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.526659 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.526662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.528119 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.528131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.528844 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.528848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.529104 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.529892 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:09.529911 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:09.532995 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:09.535711 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.535714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.535717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.535966 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.536284 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:09.536289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:09.536294 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:09.536396 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:09.536906 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:09.536909 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:09.536998 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.537107 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.537257 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1412,"mutex_wait_us":378,"spinlock_wait_cycles":3840,"thread_start_us":136,"threads_started":1}
I20260808 15:52:09.540980 8952 heartbeater.cc:511] Master 127.30.76.125:39677 requested a full tablet report, sending...
I20260808 15:52:09.541695 8949 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45997
I20260808 15:52:09.541841 8951 heartbeater.cc:347] Connected to a master server at 127.30.76.124:34473
I20260808 15:52:09.541926 8951 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.542102 8951 heartbeater.cc:511] Master 127.30.76.124:34473 requested a full tablet report, sending...
I20260808 15:52:09.542331 8065 ts_manager.cc:194] Registered new tserver with Master: ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171)
I20260808 15:52:09.542608 8949 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:09.542791 8329 ts_manager.cc:194] Registered new tserver with Master: ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171)
I20260808 15:52:09.542814 8949 heartbeater.cc:511] Master 127.30.76.126:45997 requested a full tablet report, sending...
I20260808 15:52:09.543516 7188 ts_manager.cc:194] Re-registered known tserver with Master: ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171)
I20260808 15:52:09.544340 7188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:53823
I20260808 15:52:09.546602 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.526758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.30.76.126:45997)
0808 15:52:09.527018 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.527023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.527036 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:09.527072 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.527075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.527116 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.528219 (+ 1103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.528326 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.529069 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.530271 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.530705 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.530859 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.532715 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.532720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.532723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.532749 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.544531 (+ 11782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.545259 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.545262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.545265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.545359 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.545660 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.545663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.545665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.545881 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.546077 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.546300 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.546435 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260808 15:52:09.547348 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.527145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.30.76.124:34473)
0808 15:52:09.532210 (+ 5065us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.532214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.532558 (+ 344us) server_negotiation.cc:407] Connection header received
0808 15:52:09.532599 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.532601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.532668 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.532755 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.533987 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.534872 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.538137 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.538512 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.538614 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.540758 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.540763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.540765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.540792 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.545186 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.545667 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.545671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.545673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.545759 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.546119 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.546123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.546124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.546369 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.546490 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.547030 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.547170 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4989,"thread_start_us":85,"threads_started":1}
I20260808 15:52:09.547567 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.526514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.30.76.125:39677)
0808 15:52:09.535120 (+ 8606us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.535125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.535140 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:09.535174 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.535177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.535220 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.537141 (+ 1921us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.537246 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.537992 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.540074 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.540413 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.540858 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.542067 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.542071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.542073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.542096 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.543903 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.544380 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.544384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.544387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.544481 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.546613 (+ 2132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.546616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.546618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.546881 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.547013 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.547256 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.547384 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8516,"thread_start_us":108,"threads_started":1}
I20260808 15:52:09.549746 7188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:09.552711 7188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:09.555181 8772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: NON_VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: true } } }
I20260808 15:52:09.555342 8772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:09.559386 8981 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: attempting to promote NON_VOTER ba77ae8813504150b39040df54feaac4 to VOTER
I20260808 15:52:09.559767 8981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:09.560848 8339 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 LEARNER]: Refusing update from remote peer 37ae4ca4d61d4221ab26b3f24e6dd70f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 15:52:09.561012 8075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Refusing update from remote peer 37ae4ca4d61d4221ab26b3f24e6dd70f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 15:52:09.561414 8772 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 15:52:09.561728 8981 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 15:52:09.563310 8981 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ba77ae8813504150b39040df54feaac4 (127.30.76.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.563984 8075 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ba77ae8813504150b39040df54feaac4 (127.30.76.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.564728 8979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.564772 8981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.564857 8979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:09.564877 8981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:09.565011 8980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.565104 8980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:09.565613 8339 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ba77ae8813504150b39040df54feaac4 (127.30.76.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.566039 8994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.566174 8994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:09.567972 8381 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:09.568089 8381 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:09.574195 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.566962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44171 (local address 127.0.0.1:35964)
0808 15:52:09.568461 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.568474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.568570 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.569739 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.569742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.569761 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:09.569848 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.569854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.571301 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.571305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.572421 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.572432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.573793 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.573796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.573885 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.573920 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.573981 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.574029 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"mutex_wait_us":44,"thread_start_us":115,"threads_started":1}
I20260808 15:52:09.574234 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.567986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.30.76.65:46523)
0808 15:52:09.568822 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.568836 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.569039 (+ 203us) server_negotiation.cc:407] Connection header received
0808 15:52:09.569182 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.569187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.569380 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.569834 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:09.569979 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.571167 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.572710 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.573553 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.573669 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.573836 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.573962 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.574059 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":68,"threads_started":1}
I20260808 15:52:09.574599 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.566708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:37185 (local address 127.0.0.1:34632)
0808 15:52:09.567593 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.567608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.567696 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.569188 (+ 1492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.569192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.569210 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:09.569333 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.569340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.570809 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.570814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.571822 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.571834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.574002 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.574006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.574089 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.574112 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.574164 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.574242 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774}
I20260808 15:52:09.574738 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.566863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.30.76.66:37185)
0808 15:52:09.568749 (+ 1886us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.568763 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.568788 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:52:09.568870 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.568874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.569043 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.569221 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:09.569427 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.570699 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.571964 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.572776 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.574338 (+ 1562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.574478 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.574527 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.574591 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1776,"thread_start_us":111,"threads_started":1}
I20260808 15:52:09.574203 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.567299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:46523 (local address 127.0.0.1:49236)
0808 15:52:09.568937 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.568947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.569060 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.569580 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.569582 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.569601 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:09.569723 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.569730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.571316 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.571319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.572559 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.572571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.573688 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.573693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.573809 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.573839 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.573924 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.574006 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1592,"mutex_wait_us":100,"thread_start_us":71,"threads_started":1}
I20260808 15:52:09.575145 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.567148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.30.76.67:44171)
0808 15:52:09.569309 (+ 2161us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.569320 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.569340 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:52:09.569409 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.569414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.569583 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.569988 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:09.570121 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.571186 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.572894 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.573693 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.573787 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.573901 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.574914 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.575010 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2065,"thread_start_us":119,"threads_started":1}
I20260808 15:52:09.577636 8857 tablet_service.cc:1467] Processing CreateTablet for tablet 5778454201f246fab1c001b7bfa4daf9 (DEFAULT_TABLE table=test-table [id=d701539dfdac4f10a963ab3c54003d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:09.577669 8664 tablet_service.cc:1467] Processing CreateTablet for tablet 5778454201f246fab1c001b7bfa4daf9 (DEFAULT_TABLE table=test-table [id=d701539dfdac4f10a963ab3c54003d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:09.577665 8479 tablet_service.cc:1467] Processing CreateTablet for tablet 5778454201f246fab1c001b7bfa4daf9 (DEFAULT_TABLE table=test-table [id=d701539dfdac4f10a963ab3c54003d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:09.578580 8857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5778454201f246fab1c001b7bfa4daf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:09.578711 8479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5778454201f246fab1c001b7bfa4daf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:09.578711 8664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5778454201f246fab1c001b7bfa4daf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:09.585410 9004 tablet_bootstrap.cc:492] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Bootstrap starting.
I20260808 15:52:09.586642 9005 tablet_bootstrap.cc:492] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Bootstrap starting.
I20260808 15:52:09.587491 9006 tablet_bootstrap.cc:492] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Bootstrap starting.
I20260808 15:52:09.587858 9004 tablet_bootstrap.cc:654] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:09.588724 9005 tablet_bootstrap.cc:654] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:09.588878 9004 log.cc:824] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:09.589527 9005 log.cc:824] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:09.589753 9006 tablet_bootstrap.cc:654] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:09.590530 9006 log.cc:824] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:09.591181 9004 tablet_bootstrap.cc:492] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: No bootstrap required, opened a new log
I20260808 15:52:09.591368 9005 tablet_bootstrap.cc:492] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: No bootstrap required, opened a new log
I20260808 15:52:09.591408 9004 ts_tablet_manager.cc:1397] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:52:09.591589 9005 ts_tablet_manager.cc:1397] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 15:52:09.592509 9006 tablet_bootstrap.cc:492] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: No bootstrap required, opened a new log
I20260808 15:52:09.592679 9006 ts_tablet_manager.cc:1397] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:52:09.594867 9005 raft_consensus.cc:348] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.595098 9005 raft_consensus.cc:374] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:09.595152 9005 raft_consensus.cc:729] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef601be8bcf41b887bff3d1e29f1af4, State: Initialized, Role: FOLLOWER
I20260808 15:52:09.595664 9005 consensus_queue.cc:260] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.596259 8575 heartbeater.cc:503] Master 127.30.76.126:45997 was elected leader, sending a full tablet report...
I20260808 15:52:09.596172 9004 raft_consensus.cc:348] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.596410 9004 raft_consensus.cc:374] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:09.596464 9004 raft_consensus.cc:729] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea97b41c75344804ae9678ca333e2d65, State: Initialized, Role: FOLLOWER
I20260808 15:52:09.596627 9005 ts_tablet_manager.cc:1428] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:52:09.596886 9004 consensus_queue.cc:260] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.597805 9004 ts_tablet_manager.cc:1428] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:52:09.599133 8949 heartbeater.cc:503] Master 127.30.76.126:45997 was elected leader, sending a full tablet report...
I20260808 15:52:09.600905 9006 raft_consensus.cc:348] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.601176 9006 raft_consensus.cc:374] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:09.601235 9006 raft_consensus.cc:729] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413997d0e55e42aa8fc81c653ddd8842, State: Initialized, Role: FOLLOWER
I20260808 15:52:09.601667 9006 consensus_queue.cc:260] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.602370 8764 heartbeater.cc:503] Master 127.30.76.126:45997 was elected leader, sending a full tablet report...
I20260808 15:52:09.602679 9006 ts_tablet_manager.cc:1428] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260808 15:52:09.760299 9012 raft_consensus.cc:482] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:09.760622 9012 raft_consensus.cc:504] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.761953 9012 leader_election.cc:288] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171), 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
I20260808 15:52:09.766014 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.762722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:38489 (local address 127.30.76.65:46523)
0808 15:52:09.762894 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.762901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.762917 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:09.762972 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.762974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.763010 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.763096 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:09.763379 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.763956 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.765054 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.765452 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.765583 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.765650 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.765817 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.765887 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 15:52:09.766031 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.762558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:46523 (local address 127.30.76.66:38489)
0808 15:52:09.762710 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.762728 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.762828 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.763110 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.763112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.763129 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:09.763258 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.763263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.764074 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.764078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.764924 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.764934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.765577 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.765580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.765681 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.765709 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.765800 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.765876 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":44}
I20260808 15:52:09.766173 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.762395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:49929 (local address 127.30.76.67:44171)
0808 15:52:09.762557 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.762562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.762581 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:52:09.762751 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.762755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.762816 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.762926 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:09.763217 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.763891 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.765072 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.765524 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.765676 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.765730 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.765918 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.765997 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:52:09.766179 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.762272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44171 (local address 127.30.76.66:49929)
0808 15:52:09.762433 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:09.762453 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:09.762585 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:09.762942 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:09.762945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:09.762964 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:09.763090 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.763097 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.764016 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.764020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.764928 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:09.764938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.765666 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.765670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:09.765772 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.765802 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 15:52:09.765899 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 15:52:09.765976 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":31}
I20260808 15:52:09.766834 8883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5778454201f246fab1c001b7bfa4daf9" candidate_uuid: "413997d0e55e42aa8fc81c653ddd8842" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea97b41c75344804ae9678ca333e2d65" is_pre_election: true
I20260808 15:52:09.766826 8505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5778454201f246fab1c001b7bfa4daf9" candidate_uuid: "413997d0e55e42aa8fc81c653ddd8842" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" is_pre_election: true
I20260808 15:52:09.767145 8883 raft_consensus.cc:2454] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413997d0e55e42aa8fc81c653ddd8842 in term 0.
I20260808 15:52:09.767180 8505 raft_consensus.cc:2454] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413997d0e55e42aa8fc81c653ddd8842 in term 0.
I20260808 15:52:09.767700 8607 leader_election.cc:302] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413997d0e55e42aa8fc81c653ddd8842, ea97b41c75344804ae9678ca333e2d65; no voters:
I20260808 15:52:09.768213 9012 raft_consensus.cc:2781] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:09.768297 9012 raft_consensus.cc:482] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:09.768332 9012 raft_consensus.cc:3037] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:09.769761 9012 raft_consensus.cc:504] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.770190 9012 leader_election.cc:288] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [CANDIDATE]: Term 1 election: Requested vote from peers ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171), 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
W20260808 15:52:09.770514 8953 tablet.cc:2369] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:09.770629 8883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5778454201f246fab1c001b7bfa4daf9" candidate_uuid: "413997d0e55e42aa8fc81c653ddd8842" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea97b41c75344804ae9678ca333e2d65"
I20260808 15:52:09.770744 8505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5778454201f246fab1c001b7bfa4daf9" candidate_uuid: "413997d0e55e42aa8fc81c653ddd8842" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef601be8bcf41b887bff3d1e29f1af4"
I20260808 15:52:09.770782 8883 raft_consensus.cc:3037] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:09.770874 8505 raft_consensus.cc:3037] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:09.771962 8883 raft_consensus.cc:2454] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413997d0e55e42aa8fc81c653ddd8842 in term 1.
I20260808 15:52:09.772238 8505 raft_consensus.cc:2454] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413997d0e55e42aa8fc81c653ddd8842 in term 1.
I20260808 15:52:09.772284 8607 leader_election.cc:302] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413997d0e55e42aa8fc81c653ddd8842, ea97b41c75344804ae9678ca333e2d65; no voters:
I20260808 15:52:09.772476 9012 raft_consensus.cc:2781] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:09.772660 9012 raft_consensus.cc:686] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 LEADER]: Becoming Leader. State: Replica: 413997d0e55e42aa8fc81c653ddd8842, State: Running, Role: LEADER
I20260808 15:52:09.773002 9012 consensus_queue.cc:237] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } }
I20260808 15:52:09.776839 7189 catalog_manager.cc:5495] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66). New cstate: current_term: 1 leader_uuid: "413997d0e55e42aa8fc81c653ddd8842" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } health_report { overall_health: UNKNOWN } } }
W20260808 15:52:09.795398 8767 tablet.cc:2369] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:09.813047 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.809295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.30.76.66:37185)
0808 15:52:09.809455 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.809461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.809476 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:09.809578 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.809580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.809645 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.809759 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:09.810016 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.810649 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.811519 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.811780 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.811896 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.812235 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:09.812643 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:09.812783 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.812846 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.812902 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:52:09.818744 8505 raft_consensus.cc:1261] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Refusing update from remote peer 413997d0e55e42aa8fc81c653ddd8842: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:09.818743 8883 raft_consensus.cc:1261] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Refusing update from remote peer 413997d0e55e42aa8fc81c653ddd8842: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:09.819464 9012 consensus_queue.cc:1035] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:09.819667 9015 consensus_queue.cc:1035] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:09.828981 9022 mvcc.cc:205] Tried to move back new op lower bound from 7316292934930411520 to 7316292934753349632. Current Snapshot: MvccSnapshot[applied={T|T < 7316292934930411520}]
I20260808 15:52:09.839464 7188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:09.839684 7188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:09.845319 7188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37ae4ca4d61d4221ab26b3f24e6dd70f: Sending DeleteTablet for 3 replicas of tablet 5778454201f246fab1c001b7bfa4daf9
I20260808 15:52:09.846531 8664 tablet_service.cc:1514] Processing DeleteTablet for tablet 5778454201f246fab1c001b7bfa4daf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:09 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:34632
I20260808 15:52:09.846765 8857 tablet_service.cc:1514] Processing DeleteTablet for tablet 5778454201f246fab1c001b7bfa4daf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:09 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:35964
I20260808 15:52:09.846916 8479 tablet_service.cc:1514] Processing DeleteTablet for tablet 5778454201f246fab1c001b7bfa4daf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:09 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:49236
I20260808 15:52:09.847266 9037 tablet_replica.cc:330] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: stopping tablet replica
I20260808 15:52:09.847719 9037 raft_consensus.cc:2229] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:09.848174 9037 raft_consensus.cc:2258] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:09.848273 9038 tablet_replica.cc:330] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: stopping tablet replica
I20260808 15:52:09.848577 9038 raft_consensus.cc:2229] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:09.848915 9038 raft_consensus.cc:2258] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:09.849078 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 7134
I20260808 15:52:09.849257 9036 tablet_replica.cc:330] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: stopping tablet replica
I20260808 15:52:09.853365 9036 raft_consensus.cc:2229] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:09.853683 9036 raft_consensus.cc:2258] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:09.856307 9036 ts_tablet_manager.cc:1905] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:09.856402 9037 ts_tablet_manager.cc:1905] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:09.859079 9036 ts_tablet_manager.cc:1918] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:09.859079 9037 ts_tablet_manager.cc:1918] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:09.859190 9036 log.cc:1196] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/5778454201f246fab1c001b7bfa4daf9
I20260808 15:52:09.859190 9037 log.cc:1196] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/5778454201f246fab1c001b7bfa4daf9
I20260808 15:52:09.859566 9037 ts_tablet_manager.cc:1939] T 5778454201f246fab1c001b7bfa4daf9 P ea97b41c75344804ae9678ca333e2d65: Deleting consensus metadata
I20260808 15:52:09.859580 9036 ts_tablet_manager.cc:1939] T 5778454201f246fab1c001b7bfa4daf9 P 413997d0e55e42aa8fc81c653ddd8842: Deleting consensus metadata
I20260808 15:52:09.866555 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.847841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56160 (local address 127.30.76.125:39677)
0808 15:52:09.848107 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.848115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.848832 (+ 717us) server_negotiation.cc:407] Connection header received
0808 15:52:09.849046 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.849049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.849111 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.849172 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.856886 (+ 7714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.857749 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.859201 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.859672 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.859773 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.862200 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.862208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.862214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.862251 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.864245 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.864827 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.864833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.864838 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.864940 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.865290 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.865296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.865301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.865551 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.865666 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.866198 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.866360 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 15:52:09.867646 8605 connection.cc:535] server connection from 127.0.0.1:34632 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 15:52:09.868781 8801 connection.cc:441] server connection from 127.0.0.1:35964 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35964 (request call id 1) could send its response
I20260808 15:52:09.870903 9038 ts_tablet_manager.cc:1905] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:09.874220 9038 ts_tablet_manager.cc:1918] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:09.874332 9038 log.cc:1196] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/5778454201f246fab1c001b7bfa4daf9
I20260808 15:52:09.875164 9038 ts_tablet_manager.cc:1939] T 5778454201f246fab1c001b7bfa4daf9 P 9ef601be8bcf41b887bff3d1e29f1af4: Deleting consensus metadata
W20260808 15:52:09.876581 8426 connection.cc:441] server connection from 127.0.0.1:49236 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49236 (request call id 1) could send its response
I20260808 15:52:09.879664 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.848223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48778 (local address 127.30.76.124:34473)
0808 15:52:09.848347 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.848352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.868574 (+ 20222us) server_negotiation.cc:407] Connection header received
0808 15:52:09.868761 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.868764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.868822 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.868874 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.870093 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.870850 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.872094 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.872452 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.872532 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.874465 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.874470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.874473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.874502 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.877610 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.878142 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.878146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.878147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.878269 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.878626 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.878630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.878631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.878862 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.878983 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.879340 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.879472 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:09.880352 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.869560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56162 (local address 127.30.76.125:39677)
0808 15:52:09.869665 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.869669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.870661 (+ 992us) server_negotiation.cc:407] Connection header received
0808 15:52:09.870823 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.870826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.870867 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.870914 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.872001 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.872773 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.874057 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.874378 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.874452 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.876264 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.876268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.876271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.876299 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.877994 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.878525 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.878528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.878530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.878620 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.878920 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.878923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.878925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.879135 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.879257 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.880077 (+ 820us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.880208 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:52:09.885279 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:09.875345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.30.76.124:34473)
0808 15:52:09.875469 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:09.875473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:09.876600 (+ 1127us) server_negotiation.cc:407] Connection header received
0808 15:52:09.876816 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:09.876820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:09.876863 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:09.876912 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:09.877683 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.878498 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.879838 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:09.880183 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:09.880261 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:09.882205 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:09.882208 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:09.882211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:09.882242 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:09.883741 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.884210 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.884212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.884214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.884283 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:09.884556 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:09.884558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:09.884559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:09.884712 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:09.884808 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:09.885048 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 15:52:09.885139 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:52:10.832988 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:10.832530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:43337)
0808 15:52:10.832805 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:10.832892 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
W20260808 15:52:10.833534 8764 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:45997 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:10.834070 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:10.833666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:45689)
0808 15:52:10.833869 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:10.833974 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":55,"threads_started":1}
W20260808 15:52:10.834410 8949 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:45997 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:10.837095 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:10.836672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:39075)
0808 15:52:10.836907 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:10.837002 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
W20260808 15:52:10.837423 8575 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:45997 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:11.344188 9055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 37ae4ca4d61d4221ab26b3f24e6dd70f)
I20260808 15:52:11.344383 9056 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 37ae4ca4d61d4221ab26b3f24e6dd70f)
I20260808 15:52:11.344415 9055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:11.344600 9056 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:11.345858 9056 leader_election.cc:288] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997), 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125:39677)
I20260808 15:52:11.345865 9055 leader_election.cc:288] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997), ba77ae8813504150b39040df54feaac4 (127.30.76.124:34473)
I20260808 15:52:11.346361 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.345619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54360)
0808 15:52:11.346167 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.346245 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":71,"threads_started":1}
I20260808 15:52:11.346572 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.345672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54358)
0808 15:52:11.346395 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.346478 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":293,"thread_start_us":116,"threads_started":1}
W20260808 15:52:11.346643 8044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
W20260808 15:52:11.346735 8307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:11.347685 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.347428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54382)
0808 15:52:11.347565 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.347617 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:52:11.347865 8044 leader_election.cc:334] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:11.348155 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.347668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54388)
0808 15:52:11.348034 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.348091 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 15:52:11.348258 8307 leader_election.cc:334] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:11.350030 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.345673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34473 (local address 127.0.0.1:52900)
0808 15:52:11.346296 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.346308 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:11.346409 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:11.346737 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:11.346739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:11.346756 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:11.346861 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:11.346871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.347983 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.347987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:11.348868 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:11.348878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.349525 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.349532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:11.349687 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:11.349704 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:52:11.349799 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 15:52:11.349871 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":25,"thread_start_us":70,"threads_started":1}
I20260808 15:52:11.350032 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.345668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:39677 (local address 127.0.0.1:37122)
0808 15:52:11.346088 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.346107 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:11.346215 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:11.346630 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:11.346635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:11.346657 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:11.346784 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:11.346792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.347840 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.347844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:11.348859 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:11.348869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.349508 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.349512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:11.349681 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:11.349700 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 15:52:11.349791 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 15:52:11.349866 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":48,"thread_start_us":60,"threads_started":1}
I20260808 15:52:11.350036 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.345935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37122 (local address 127.30.76.125:39677)
0808 15:52:11.346167 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:11.346173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:11.346191 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:52:11.346353 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:11.346356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:11.346422 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:11.346500 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:11.346923 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.347715 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.348985 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.349378 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.349498 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:11.349614 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:11.349809 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 15:52:11.349891 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":165,"spinlock_wait_cycles":2816,"thread_start_us":54,"threads_started":1}
I20260808 15:52:11.350036 9063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.345869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.30.76.124:34473)
0808 15:52:11.346286 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:11.346291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:11.346502 (+ 211us) server_negotiation.cc:407] Connection header received
0808 15:52:11.346554 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:11.346556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:11.346597 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:11.346676 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:11.347066 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.347849 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.348995 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:11.349395 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:11.349511 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:11.349624 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:11.349818 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 15:52:11.349901 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":286,"server-negotiator.queue_time_us":341,"thread_start_us":50,"threads_started":1}
I20260808 15:52:11.350601 8339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86de954cd47e4e65a093fefa1b027a47" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ba77ae8813504150b39040df54feaac4" is_pre_election: true
I20260808 15:52:11.350601 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba77ae8813504150b39040df54feaac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86de954cd47e4e65a093fefa1b027a47" is_pre_election: true
I20260808 15:52:11.350803 8339 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86de954cd47e4e65a093fefa1b027a47 in term 1.
I20260808 15:52:11.350803 8075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba77ae8813504150b39040df54feaac4 in term 1.
I20260808 15:52:11.351146 8308 leader_election.cc:302] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86de954cd47e4e65a093fefa1b027a47, ba77ae8813504150b39040df54feaac4; no voters: 37ae4ca4d61d4221ab26b3f24e6dd70f
I20260808 15:52:11.351140 8047 leader_election.cc:302] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86de954cd47e4e65a093fefa1b027a47, ba77ae8813504150b39040df54feaac4; no voters: 37ae4ca4d61d4221ab26b3f24e6dd70f
I20260808 15:52:11.351352 9055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 15:52:11.351379 9056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 15:52:11.351408 9055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 37ae4ca4d61d4221ab26b3f24e6dd70f)
I20260808 15:52:11.351428 9056 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 37ae4ca4d61d4221ab26b3f24e6dd70f)
I20260808 15:52:11.351449 9055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:52:11.351468 9056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:52:11.352465 9055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:11.352468 9056 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:11.352798 9056 leader_election.cc:288] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 2 election: Requested vote from peers 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997), 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125:39677)
I20260808 15:52:11.352800 9055 leader_election.cc:288] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [CANDIDATE]: Term 2 election: Requested vote from peers 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997), ba77ae8813504150b39040df54feaac4 (127.30.76.124:34473)
I20260808 15:52:11.353138 8339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86de954cd47e4e65a093fefa1b027a47" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ba77ae8813504150b39040df54feaac4"
I20260808 15:52:11.353271 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.353024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54404)
0808 15:52:11.353152 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.353209 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 15:52:11.353358 8339 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 86de954cd47e4e65a093fefa1b027a47 in current term 2: Already voted for candidate ba77ae8813504150b39040df54feaac4 in this term.
I20260808 15:52:11.353420 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba77ae8813504150b39040df54feaac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86de954cd47e4e65a093fefa1b027a47"
I20260808 15:52:11.353579 8075 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba77ae8813504150b39040df54feaac4 in current term 2: Already voted for candidate 86de954cd47e4e65a093fefa1b027a47 in this term.
I20260808 15:52:11.353143 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.352887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54394)
0808 15:52:11.353030 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.353079 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 15:52:11.354082 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.353746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54420)
0808 15:52:11.353933 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.354007 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260808 15:52:11.354210 8044 leader_election.cc:334] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:11.354306 8044 leader_election.cc:302] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86de954cd47e4e65a093fefa1b027a47; no voters: 37ae4ca4d61d4221ab26b3f24e6dd70f, ba77ae8813504150b39040df54feaac4
I20260808 15:52:11.354365 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.354176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54426)
0808 15:52:11.354264 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.354310 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 15:52:11.354465 9055 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260808 15:52:11.354629 8307 leader_election.cc:334] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:11.354699 8307 leader_election.cc:302] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba77ae8813504150b39040df54feaac4; no voters: 37ae4ca4d61d4221ab26b3f24e6dd70f, 86de954cd47e4e65a093fefa1b027a47
I20260808 15:52:11.354822 9056 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 15:52:11.834712 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.834202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:46501)
0808 15:52:11.834498 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.834574 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":108,"threads_started":1}
I20260808 15:52:11.835314 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.834966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:50515)
0808 15:52:11.835181 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.835239 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260808 15:52:11.838443 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:11.837958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:49687)
0808 15:52:11.838277 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:11.838338 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":85,"threads_started":1}
I20260808 15:52:12.836055 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:12.835520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:37953)
0808 15:52:12.835866 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:12.835941 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":92,"threads_started":1}
I20260808 15:52:12.836324 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:12.835935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:59655)
0808 15:52:12.836157 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:12.836223 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260808 15:52:12.839520 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:12.839065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:33189)
0808 15:52:12.839361 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:12.839425 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":61,"threads_started":1}
I20260808 15:52:13.086308 9080 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:13.086455 9080 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:13.087086 9080 leader_election.cc:288] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997), 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125:39677)
I20260808 15:52:13.087685 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba77ae8813504150b39040df54feaac4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86de954cd47e4e65a093fefa1b027a47" is_pre_election: true
I20260808 15:52:13.087841 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.087322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54634)
0808 15:52:13.087663 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.087720 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":81,"threads_started":1}
I20260808 15:52:13.087831 8075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba77ae8813504150b39040df54feaac4 in term 2.
I20260808 15:52:13.088228 8308 leader_election.cc:302] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86de954cd47e4e65a093fefa1b027a47, ba77ae8813504150b39040df54feaac4; no voters:
I20260808 15:52:13.088409 9080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 15:52:13.088462 9080 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:13.088490 9080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 2 FOLLOWER]: Advancing to term 3
I20260808 15:52:13.088819 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.088578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54646)
0808 15:52:13.088692 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.088750 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 15:52:13.088989 8307 leader_election.cc:334] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:13.089680 9080 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:13.090020 9080 leader_election.cc:288] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 3 election: Requested vote from peers 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997), 86de954cd47e4e65a093fefa1b027a47 (127.30.76.125:39677)
I20260808 15:52:13.090337 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.090133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54650)
0808 15:52:13.090235 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.090286 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 15:52:13.090401 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba77ae8813504150b39040df54feaac4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86de954cd47e4e65a093fefa1b027a47"
I20260808 15:52:13.090545 8075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 2 FOLLOWER]: Advancing to term 3
I20260808 15:52:13.090998 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.090810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54664)
0808 15:52:13.090908 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.090951 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:52:13.091109 8307 leader_election.cc:334] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
I20260808 15:52:13.091717 8075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba77ae8813504150b39040df54feaac4 in term 3.
I20260808 15:52:13.092000 8308 leader_election.cc:302] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86de954cd47e4e65a093fefa1b027a47, ba77ae8813504150b39040df54feaac4; no voters: 37ae4ca4d61d4221ab26b3f24e6dd70f
I20260808 15:52:13.092206 9080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 15:52:13.092403 9080 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [term 3 LEADER]: Becoming Leader. State: Replica: ba77ae8813504150b39040df54feaac4, State: Running, Role: LEADER
I20260808 15:52:13.092669 9080 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } }
I20260808 15:52:13.094131 9083 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba77ae8813504150b39040df54feaac4. Latest consensus state: current_term: 3 leader_uuid: "ba77ae8813504150b39040df54feaac4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:13.094264 9083 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:13.094579 9085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:13.097108 9085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:13.097630 9085 catalog_manager.cc:1259] Loaded cluster ID: 2dd02fd64a094281b6e2bfa962b0b00b
I20260808 15:52:13.097682 9085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:13.098306 9085 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:13.098812 9085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Loaded TSK: 0
I20260808 15:52:13.099570 9085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:52:13.186486 8328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:13.189682 8328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:13.192781 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.192531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54704)
0808 15:52:13.192654 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.192714 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:52:13.192852 8075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [term 3 FOLLOWER]: Refusing update from remote peer ba77ae8813504150b39040df54feaac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 15:52:13.193349 9083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 15:52:13.193532 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.193272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.0.0.1:54708)
0808 15:52:13.193382 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.193443 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:52:13.193666 8307 consensus_peers.cc:487] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 -> Peer 37ae4ca4d61d4221ab26b3f24e6dd70f (127.30.76.126:45997): Couldn't send request to peer 37ae4ca4d61d4221ab26b3f24e6dd70f. Status: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 15:52:13.194749 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba77ae8813504150b39040df54feaac4. Latest consensus state: current_term: 3 leader_uuid: "ba77ae8813504150b39040df54feaac4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:13.194881 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:13.195981 9083 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ba77ae8813504150b39040df54feaac4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:13.196095 9083 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:13.196295 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ba77ae8813504150b39040df54feaac4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "37ae4ca4d61d4221ab26b3f24e6dd70f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45997 } } peers { permanent_uuid: "86de954cd47e4e65a093fefa1b027a47" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 39677 } attrs { promote: false } } peers { permanent_uuid: "ba77ae8813504150b39040df54feaac4" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34473 } attrs { promote: false } } }
I20260808 15:52:13.196458 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de954cd47e4e65a093fefa1b027a47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:52:13.206583 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.201985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:37185 (local address 127.0.0.1:35674)
0808 15:52:13.202157 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.202173 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:13.202262 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:13.203166 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:13.203169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:13.203188 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:13.203321 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.203327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.204423 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.204426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.205526 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.205535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.206173 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.206176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.206269 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.206300 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:52:13.206378 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:52:13.206449 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"spinlock_wait_cycles":3328}
I20260808 15:52:13.206614 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.202116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.30.76.66:37185)
0808 15:52:13.202642 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:13.202647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:13.202661 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:52:13.202717 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:13.202723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:13.202779 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:13.203144 (+ 365us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:13.203454 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.204294 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.205661 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.206050 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.206168 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.206231 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:13.206399 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 15:52:13.206482 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":82,"threads_started":1}
I20260808 15:52:13.207448 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.202818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44171 (local address 127.0.0.1:59412)
0808 15:52:13.203141 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.203155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:13.203235 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:13.203725 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:13.203728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:13.203745 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:13.203832 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.203837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.204840 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.204843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.206208 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.206219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.207083 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.207087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.207179 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.207205 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 15:52:13.207254 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 15:52:13.207318 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":70,"thread_start_us":100,"threads_started":1}
I20260808 15:52:13.207397 8664 tablet_service.cc:1467] Processing CreateTablet for tablet 4c6e5127a11443299d8a49f2dc206545 (DEFAULT_TABLE table=test-table [id=7d653310f8bd43e381bcaa46f58dc29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:13.207655 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.202631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.30.76.65:46523)
0808 15:52:13.203025 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:13.203030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:13.203434 (+ 404us) server_negotiation.cc:407] Connection header received
0808 15:52:13.203492 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:13.203494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:13.203609 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:13.203717 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:13.204111 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.204680 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.206358 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.206993 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.207099 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.207167 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:13.207436 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 15:52:13.207509 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260808 15:52:13.207653 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.202493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:46523 (local address 127.0.0.1:40574)
0808 15:52:13.203068 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.203080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:13.203204 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:13.203936 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:13.203937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:13.203949 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:13.204000 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.204003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.204930 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.204933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.206208 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.206219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.207221 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.207224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.207300 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.207318 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 15:52:13.207405 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 15:52:13.207479 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":71,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260808 15:52:13.207844 8664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c6e5127a11443299d8a49f2dc206545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:13.208348 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.202955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.30.76.67:44171)
0808 15:52:13.203454 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:13.203459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:13.203472 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:13.203566 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:13.203569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:13.203617 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:13.203700 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:13.203955 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.204699 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.206316 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.206960 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.207973 (+ 1013us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.208033 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:13.208070 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 15:52:13.208186 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":63,"threads_started":1}
I20260808 15:52:13.208555 8479 tablet_service.cc:1467] Processing CreateTablet for tablet 4c6e5127a11443299d8a49f2dc206545 (DEFAULT_TABLE table=test-table [id=7d653310f8bd43e381bcaa46f58dc29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:13.208832 8479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c6e5127a11443299d8a49f2dc206545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:13.208876 8857 tablet_service.cc:1467] Processing CreateTablet for tablet 4c6e5127a11443299d8a49f2dc206545 (DEFAULT_TABLE table=test-table [id=7d653310f8bd43e381bcaa46f58dc29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:13.209129 8857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c6e5127a11443299d8a49f2dc206545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:13.210240 9103 tablet_bootstrap.cc:492] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Bootstrap starting.
I20260808 15:52:13.211244 9103 tablet_bootstrap.cc:654] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:13.211700 9105 tablet_bootstrap.cc:492] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: Bootstrap starting.
I20260808 15:52:13.212145 9108 tablet_bootstrap.cc:492] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Bootstrap starting.
I20260808 15:52:13.212702 9103 tablet_bootstrap.cc:492] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: No bootstrap required, opened a new log
I20260808 15:52:13.212746 9105 tablet_bootstrap.cc:654] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:13.212810 9103 ts_tablet_manager.cc:1397] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:13.212949 9108 tablet_bootstrap.cc:654] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:13.213253 9103 raft_consensus.cc:348] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.213374 9103 raft_consensus.cc:374] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:13.213407 9103 raft_consensus.cc:729] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413997d0e55e42aa8fc81c653ddd8842, State: Initialized, Role: FOLLOWER
I20260808 15:52:13.213560 9103 consensus_queue.cc:260] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.213977 9103 ts_tablet_manager.cc:1428] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:13.214064 9105 tablet_bootstrap.cc:492] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: No bootstrap required, opened a new log
I20260808 15:52:13.214154 9105 ts_tablet_manager.cc:1397] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 15:52:13.214614 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.214294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:57447)
0808 15:52:13.214476 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.214537 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 15:52:13.214612 9105 raft_consensus.cc:348] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.214725 9105 raft_consensus.cc:374] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:13.214761 9105 raft_consensus.cc:729] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef601be8bcf41b887bff3d1e29f1af4, State: Initialized, Role: FOLLOWER
I20260808 15:52:13.214888 9105 consensus_queue.cc:260] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.215255 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.215040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:56237)
0808 15:52:13.215147 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.215202 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 15:52:13.215250 9105 ts_tablet_manager.cc:1428] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 15:52:13.217383 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.217091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:37781)
0808 15:52:13.217241 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.217296 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 15:52:13.218108 9108 tablet_bootstrap.cc:492] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: No bootstrap required, opened a new log
I20260808 15:52:13.218120 8765 heartbeater.cc:503] Master 127.30.76.124:34473 was elected leader, sending a full tablet report...
I20260808 15:52:13.218199 9108 ts_tablet_manager.cc:1397] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 15:52:13.218721 9108 raft_consensus.cc:348] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.218822 9108 raft_consensus.cc:374] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:13.218844 9108 raft_consensus.cc:729] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea97b41c75344804ae9678ca333e2d65, State: Initialized, Role: FOLLOWER
I20260808 15:52:13.218986 9108 consensus_queue.cc:260] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.219367 9108 ts_tablet_manager.cc:1428] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 15:52:13.220233 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.219997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:44631)
0808 15:52:13.220093 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.220160 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 15:52:13.220654 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.220376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:50597)
0808 15:52:13.220530 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.220579 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 15:52:13.220805 8576 heartbeater.cc:503] Master 127.30.76.124:34473 was elected leader, sending a full tablet report...
I20260808 15:52:13.220908 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.220667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:58491)
0808 15:52:13.220801 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.220848 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 15:52:13.221112 8951 heartbeater.cc:503] Master 127.30.76.124:34473 was elected leader, sending a full tablet report...
W20260808 15:52:13.333202 8578 tablet.cc:2369] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:13.414949 9115 raft_consensus.cc:482] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:13.415146 9115 raft_consensus.cc:504] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.416298 9115 leader_election.cc:288] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185), 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
I20260808 15:52:13.420562 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.416572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:37185 (local address 127.30.76.67:47537)
0808 15:52:13.416702 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.416717 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:13.416976 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:13.417360 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:13.417363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:13.417391 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:13.417554 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.417564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.418400 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.418404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.419313 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.419323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.420063 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.420068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.420176 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.420250 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 15:52:13.420326 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 15:52:13.420413 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":35}
I20260808 15:52:13.420575 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.416689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:47537 (local address 127.30.76.66:37185)
0808 15:52:13.416855 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:13.416861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:13.416880 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:52:13.417179 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:13.417182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:13.417237 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:13.417322 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:13.417681 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.418283 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.419444 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.419918 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.420012 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.420052 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:13.420367 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 15:52:13.420448 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 15:52:13.420627 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.417107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:58827 (local address 127.30.76.65:46523)
0808 15:52:13.417246 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:13.417250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:13.417364 (+ 114us) server_negotiation.cc:407] Connection header received
0808 15:52:13.417478 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:13.417481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:13.417533 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:13.417622 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:13.417921 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.418485 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.419624 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.420046 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.420180 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.420235 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:13.420435 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 15:52:13.420510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:52:13.420601 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.416899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:46523 (local address 127.30.76.67:58827)
0808 15:52:13.417266 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.417281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:13.417376 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:13.417677 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:13.417679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:13.417698 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:13.417809 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.417815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.418621 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.418630 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.419466 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:13.419477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.420182 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.420184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:13.420283 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.420322 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 15:52:13.420403 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 15:52:13.420469 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":3,"thread_start_us":143,"threads_started":1}
I20260808 15:52:13.421341 8505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c6e5127a11443299d8a49f2dc206545" candidate_uuid: "ea97b41c75344804ae9678ca333e2d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" is_pre_election: true
I20260808 15:52:13.421343 8690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c6e5127a11443299d8a49f2dc206545" candidate_uuid: "ea97b41c75344804ae9678ca333e2d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413997d0e55e42aa8fc81c653ddd8842" is_pre_election: true
I20260808 15:52:13.421504 8505 raft_consensus.cc:2454] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea97b41c75344804ae9678ca333e2d65 in term 0.
I20260808 15:52:13.421561 8690 raft_consensus.cc:2454] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea97b41c75344804ae9678ca333e2d65 in term 0.
I20260808 15:52:13.421875 8801 leader_election.cc:302] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ef601be8bcf41b887bff3d1e29f1af4, ea97b41c75344804ae9678ca333e2d65; no voters:
I20260808 15:52:13.422158 9115 raft_consensus.cc:2781] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:13.422221 9115 raft_consensus.cc:482] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:13.422274 9115 raft_consensus.cc:3037] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:13.423585 9115 raft_consensus.cc:504] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.423959 9115 leader_election.cc:288] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [CANDIDATE]: Term 1 election: Requested vote from peers 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185), 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523)
I20260808 15:52:13.424261 8690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c6e5127a11443299d8a49f2dc206545" candidate_uuid: "ea97b41c75344804ae9678ca333e2d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413997d0e55e42aa8fc81c653ddd8842"
I20260808 15:52:13.424391 8690 raft_consensus.cc:3037] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:13.424345 8505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c6e5127a11443299d8a49f2dc206545" candidate_uuid: "ea97b41c75344804ae9678ca333e2d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef601be8bcf41b887bff3d1e29f1af4"
I20260808 15:52:13.424516 8505 raft_consensus.cc:3037] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:13.425405 8690 raft_consensus.cc:2454] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea97b41c75344804ae9678ca333e2d65 in term 1.
I20260808 15:52:13.425411 8505 raft_consensus.cc:2454] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea97b41c75344804ae9678ca333e2d65 in term 1.
I20260808 15:52:13.425686 8801 leader_election.cc:302] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413997d0e55e42aa8fc81c653ddd8842, ea97b41c75344804ae9678ca333e2d65; no voters:
I20260808 15:52:13.425886 9115 raft_consensus.cc:2781] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:13.426115 9115 raft_consensus.cc:686] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 1 LEADER]: Becoming Leader. State: Replica: ea97b41c75344804ae9678ca333e2d65, State: Running, Role: LEADER
I20260808 15:52:13.426374 9115 consensus_queue.cc:237] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } }
I20260808 15:52:13.426815 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.426587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:40773)
0808 15:52:13.426703 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.426759 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 15:52:13.428673 8328 catalog_manager.cc:5495] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea97b41c75344804ae9678ca333e2d65 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "ea97b41c75344804ae9678ca333e2d65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea97b41c75344804ae9678ca333e2d65" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44171 } health_report { overall_health: HEALTHY } } }
I20260808 15:52:13.445362 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.441717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59428 (local address 127.30.76.67:44171)
0808 15:52:13.441846 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:13.441853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:13.441868 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:13.442063 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:13.442065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:13.442117 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:13.442230 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:13.442465 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.442970 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.443868 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:13.444081 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:13.444194 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:13.444449 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:13.444957 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:13.445073 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:13.445173 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 15:52:13.445225 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:52:13.448146 8505 raft_consensus.cc:1261] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Refusing update from remote peer ea97b41c75344804ae9678ca333e2d65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:13.448349 8690 raft_consensus.cc:1261] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Refusing update from remote peer ea97b41c75344804ae9678ca333e2d65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:13.448662 9117 consensus_queue.cc:1035] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef601be8bcf41b887bff3d1e29f1af4" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:13.448920 9115 consensus_queue.cc:1035] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413997d0e55e42aa8fc81c653ddd8842" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:13.450362 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.450081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:60795)
0808 15:52:13.450226 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.450273 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 15:52:13.453388 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.453096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:48091)
0808 15:52:13.453243 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.453302 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 15:52:13.453882 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.453599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:55109)
0808 15:52:13.453743 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.453807 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 15:52:13.454164 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.453854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.65:43685)
0808 15:52:13.454022 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.454081 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 15:52:13.457100 9129 mvcc.cc:205] Tried to move back new op lower bound from 7316292949798252544 to 7316292949717004288. Current Snapshot: MvccSnapshot[applied={T|T < 7316292949798252544}]
I20260808 15:52:13.459596 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.459313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.67:36673)
0808 15:52:13.459453 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.459503 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 15:52:13.460522 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.460256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45997 (local address 127.30.76.66:52059)
0808 15:52:13.460394 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.460452 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 15:52:13.463349 8326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:13.463510 8326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:13.467748 8326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba77ae8813504150b39040df54feaac4: Sending DeleteTablet for 3 replicas of tablet 4c6e5127a11443299d8a49f2dc206545
I20260808 15:52:13.468333 8664 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c6e5127a11443299d8a49f2dc206545 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:13 UTC) from {username='oryx', principal='oryx/127.30.76.124@KRBTEST.COM'} at 127.0.0.1:35674
I20260808 15:52:13.468567 8857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c6e5127a11443299d8a49f2dc206545 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:13 UTC) from {username='oryx', principal='oryx/127.30.76.124@KRBTEST.COM'} at 127.0.0.1:59412
I20260808 15:52:13.468683 8479 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c6e5127a11443299d8a49f2dc206545 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:13 UTC) from {username='oryx', principal='oryx/127.30.76.124@KRBTEST.COM'} at 127.0.0.1:40574
I20260808 15:52:13.469086 9142 tablet_replica.cc:330] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4: stopping tablet replica
I20260808 15:52:13.469192 9138 tablet_replica.cc:330] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: stopping tablet replica
I20260808 15:52:13.469305 9138 raft_consensus.cc:2229] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:13.469307 9142 raft_consensus.cc:2229] T 4c6e5127a11443299d8a49f2dc206545 P 9ef601be8bcf41b887bff3d1e29f1af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:13.469396 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 8275
I20260808 15:52:13.469434 9138 raft_consensus.cc:2258] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:13.472097 9140 tablet_replica.cc:330] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: stopping tablet replica
I20260808 15:52:13.472241 9140 raft_consensus.cc:2229] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:13.472527 9140 raft_consensus.cc:2258] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:13.473065 9138 ts_tablet_manager.cc:1905] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:13.475580 9138 ts_tablet_manager.cc:1918] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:13.475670 9138 log.cc:1196] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/4c6e5127a11443299d8a49f2dc206545
I20260808 15:52:13.475955 9138 ts_tablet_manager.cc:1939] T 4c6e5127a11443299d8a49f2dc206545 P 413997d0e55e42aa8fc81c653ddd8842: Deleting consensus metadata
I20260808 15:52:13.477504 8309 catalog_manager.cc:4841] TS 413997d0e55e42aa8fc81c653ddd8842 (127.30.76.66:37185): tablet 4c6e5127a11443299d8a49f2dc206545 (table test-table [id=7d653310f8bd43e381bcaa46f58dc29b]) successfully deleted
I20260808 15:52:13.477732 9140 ts_tablet_manager.cc:1905] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:13.480206 9140 ts_tablet_manager.cc:1918] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:13.480301 9140 log.cc:1196] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/4c6e5127a11443299d8a49f2dc206545
I20260808 15:52:13.480583 9140 ts_tablet_manager.cc:1939] T 4c6e5127a11443299d8a49f2dc206545 P ea97b41c75344804ae9678ca333e2d65: Deleting consensus metadata
I20260808 15:52:13.481334 8309 catalog_manager.cc:4841] TS ea97b41c75344804ae9678ca333e2d65 (127.30.76.67:44171): tablet 4c6e5127a11443299d8a49f2dc206545 (table test-table [id=7d653310f8bd43e381bcaa46f58dc29b]) successfully deleted
I20260808 15:52:13.487264 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 8583
I20260808 15:52:13.491137 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:13.490683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:46523 (local address 127.0.0.1:40578)
0808 15:52:13.490939 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:13.491021 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:46523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260808 15:52:13.493054 8309 catalog_manager.cc:4584] TS 9ef601be8bcf41b887bff3d1e29f1af4 (127.30.76.65:46523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c6e5127a11443299d8a49f2dc206545: Network error: Client connection negotiation failed: client connection to 127.30.76.65:46523: connect: Connection refused (error 111)
I20260808 15:52:13.503506 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 8771
I20260808 15:52:13.514566 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 7853
I20260808 15:52:13.525484 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 8010
2026-08-08T15:52:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7681 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:13 dist-test-slave-sb2z krb5kdc[9145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:13 dist-test-slave-sb2z krb5kdc[9145](info): set up 2 sockets
Aug 08 15:52:13 dist-test-slave-sb2z krb5kdc[9145](info): commencing operation
krb5kdc: starting...
W20260808 15:52:14.743095 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:14 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab.
Entry for principal oryx/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab.
I20260808 15:52:14.871867 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:42555
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab
--principal=oryx/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:42555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:14.982954 9164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:14.983234 9164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:14.983309 9164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:14.987231 9164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:14.987314 9164 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 15:52:14.987347 9164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:14.987389 9164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:14.987426 9164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:14.987454 9164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:14.992674 9164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:42555
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.126.keytab
--principal=oryx/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:42555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:14.993655 9164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:14.994632 9164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:15.003206 9174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:15.003242 9177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.003319 9164 server_base.cc:1034] running on GCE node
W20260808 15:52:15.003242 9175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.004101 9164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:15.005160 9164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:15.006362 9164 hybrid_clock.cc:648] HybridClock initialized: now 1786204335006341 us; error 34 us; skew 500 ppm
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:15.009157 9164 init.cc:377] Logged in from keytab as oryx/127.30.76.126@KRBTEST.COM (short username oryx)
I20260808 15:52:15.010008 9164 webserver.cc:460] Webserver started at http://127.30.76.126:38533/ using document root <none> and password file <none>
I20260808 15:52:15.010569 9164 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:15.010663 9164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:15.010893 9164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:15.012953 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "00c1aa206845732a78c14a704a51d015"
server_key_iv: "2490b0f447e82d97c0cf265fde27aa23"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.013526 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "00c1aa206845732a78c14a704a51d015"
server_key_iv: "2490b0f447e82d97c0cf265fde27aa23"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.017485 9164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 15:52:15.020167 9189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.021626 9164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:15.021803 9164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "00c1aa206845732a78c14a704a51d015"
server_key_iv: "2490b0f447e82d97c0cf265fde27aa23"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.021940 9164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:15.053848 9164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:15.057156 9164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:15.057379 9164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:15.066175 9164 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:42555
I20260808 15:52:15.066188 9271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:42555 every 8 connection(s)
I20260808 15:52:15.067204 9164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:15.068089 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 9164
I20260808 15:52:15.068226 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:15.068548 31025 external_mini_cluster.cc:1428] Setting key 2aeb800a426f590052eb605a607bfa3f
I20260808 15:52:15.070470 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:15.076074 9273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Bootstrap starting.
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:15.078411 9273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:15.079362 9273 log.cc:824] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:15.081431 9273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: No bootstrap required, opened a new log
I20260808 15:52:15.084429 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.070017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.30.76.126:42555)
0808 15:52:15.070485 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.070501 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.070539 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:52:15.071293 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.071311 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.071736 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.072094 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.072994 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.074074 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.075029 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.075625 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.075806 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.078542 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.078550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.078568 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.078603 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.081161 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.081688 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.081694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.081700 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.081853 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.082240 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.082244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.082246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.082734 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.082917 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.083207 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.083511 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":136,"threads_started":1}
I20260808 15:52:15.085319 9273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 42555 } }
I20260808 15:52:15.085587 9273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:15.085673 9273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2301fa59c9a4432c9e12e3ba5a21bf7f, State: Initialized, Role: FOLLOWER
I20260808 15:52:15.086124 9273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 42555 } }
I20260808 15:52:15.086275 9273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:15.086340 9273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:15.086447 9273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:15.087394 9273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 42555 } }
I20260808 15:52:15.087739 9273 leader_election.cc:302] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2301fa59c9a4432c9e12e3ba5a21bf7f; no voters:
I20260808 15:52:15.088047 9273 leader_election.cc:288] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:15.088186 9279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:15.088402 9279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [term 1 LEADER]: Becoming Leader. State: Replica: 2301fa59c9a4432c9e12e3ba5a21bf7f, State: Running, Role: LEADER
I20260808 15:52:15.088701 9279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 42555 } }
I20260808 15:52:15.089246 9273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:15.090371 9281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 42555 } } }
I20260808 15:52:15.090512 9281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:15.090788 9282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2301fa59c9a4432c9e12e3ba5a21bf7f. Latest consensus state: current_term: 1 leader_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2301fa59c9a4432c9e12e3ba5a21bf7f" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 42555 } } }
I20260808 15:52:15.090888 9282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:15.090986 9294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:15.094103 9294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:15.099480 9294 catalog_manager.cc:1347] Generated new cluster ID: 997bdb94512742bab844432268d86b4e
I20260808 15:52:15.099582 9294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:15.115433 9294 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:15.116336 9294 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:15.123983 9294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Generated new TSK 0
I20260808 15:52:15.124603 9294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab.
Entry for principal oryx/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab.
I20260808 15:52:15.158844 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:42555
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:15.270955 9310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:15.271184 9310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:15.271227 9310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:15.275098 9310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:15.275144 9310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:15.275182 9310 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:52:15.275213 9310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:15.280062 9310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:42555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:15.280954 9310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:15.281872 9310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:15.291337 9317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:15.291347 9316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.291381 9310 server_base.cc:1034] running on GCE node
W20260808 15:52:15.291350 9320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.291875 9310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:15.292486 9310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:15.293661 9310 hybrid_clock.cc:648] HybridClock initialized: now 1786204335293630 us; error 44 us; skew 500 ppm
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:15.296622 9310 init.cc:377] Logged in from keytab as oryx/127.30.76.65@KRBTEST.COM (short username oryx)
I20260808 15:52:15.297453 9310 webserver.cc:460] Webserver started at http://127.30.76.65:40837/ using document root <none> and password file <none>
I20260808 15:52:15.298002 9310 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:15.298094 9310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:15.298324 9310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:15.300356 9310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "e8a9269d2c6044668385fd7f0081b3b7"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "8e095bee14af970f4c6e763282bc3664"
server_key_iv: "0b4d54dcbb5a44c501a109c569d8c831"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.300905 9310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "e8a9269d2c6044668385fd7f0081b3b7"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "8e095bee14af970f4c6e763282bc3664"
server_key_iv: "0b4d54dcbb5a44c501a109c569d8c831"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.304531 9310 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:15.306771 9327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.308039 9310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:15.308139 9310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "e8a9269d2c6044668385fd7f0081b3b7"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "8e095bee14af970f4c6e763282bc3664"
server_key_iv: "0b4d54dcbb5a44c501a109c569d8c831"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.308218 9310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:15.321462 9310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:15.326563 9310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:15.326779 9310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:15.328511 9310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:15.328600 9310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.328681 9310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:15.328737 9310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:15.342995 9310 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:42179
I20260808 15:52:15.343477 9487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:42179 every 8 connection(s)
I20260808 15:52:15.344269 9310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:15.347210 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.329473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:60587 (local address 127.30.76.126:42555)
0808 15:52:15.329615 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.329620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.330757 (+ 1137us) server_negotiation.cc:407] Connection header received
0808 15:52:15.332226 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.332230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.332299 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.332357 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.334306 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.335202 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.337182 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.337695 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.337802 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.343010 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.343017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.343020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.343053 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.344844 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.345521 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.345526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.345528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.345633 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.346038 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.346058 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.346060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.346261 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.346383 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.346907 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.347037 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:52:15.348233 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.330019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.30.76.65:60587)
0808 15:52:15.330578 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.330644 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.331930 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.332820 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.332832 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.333415 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:15.334095 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.334118 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.335375 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.335381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.336992 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.337008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.338068 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.338076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.338323 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.340759 (+ 2436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:15.340782 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:15.342826 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:15.344953 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.344958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.344971 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.345367 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.345740 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.345744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.345746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.345928 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.346383 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:15.346390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:15.346665 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.346845 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.347166 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
I20260808 15:52:15.348534 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 9310
I20260808 15:52:15.348692 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:15.349066 31025 external_mini_cluster.cc:1428] Setting key a42371c43e85bd2566445c18a8961c4e
I20260808 15:52:15.349870 9489 heartbeater.cc:347] Connected to a master server at 127.30.76.126:42555
I20260808 15:52:15.350052 9489 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:15.350772 9489 heartbeater.cc:511] Master 127.30.76.126:42555 requested a full tablet report, sending...
I20260808 15:52:15.353930 9211 ts_manager.cc:194] Registered new tserver with Master: e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179)
I20260808 15:52:15.355602 9211 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:60587
I20260808 15:52:15.361289 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.353437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:35899 (local address 127.30.76.126:42555)
0808 15:52:15.353565 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.353570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.353704 (+ 134us) server_negotiation.cc:407] Connection header received
0808 15:52:15.353817 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.353821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.353874 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.353957 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.354494 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.355007 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.356116 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.356431 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.356530 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.357912 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.357916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.357919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.357944 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.359652 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.360089 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.360092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.360094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.360179 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.360469 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.360472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.360474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.360649 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.360757 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.361040 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.361148 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:52:15.361303 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.353098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.30.76.65:35899)
0808 15:52:15.353627 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.353643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.353738 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.354021 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.354026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.354242 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:15.354361 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.354369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.355128 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.355131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.355983 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.355994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.356666 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.356670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.356772 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.357320 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:15.357344 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:15.357758 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:15.359760 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.359764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.359769 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.359986 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.360273 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.360276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.360278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.360369 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.360748 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:15.360751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:15.360849 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.360971 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.361107 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for oryx/127.30.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab.
Entry for principal oryx/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab.
I20260808 15:52:15.381745 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:42555
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:15.491324 9499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:15.491613 9499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:15.491688 9499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:15.495661 9499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:15.495735 9499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:15.495803 9499 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:52:15.495882 9499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:15.501174 9499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:42555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:15.502198 9499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:15.503224 9499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:15.512827 9505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:15.512822 9504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.512969 9499 server_base.cc:1034] running on GCE node
W20260808 15:52:15.512822 9509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.513422 9499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:15.514061 9499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:15.515221 9499 hybrid_clock.cc:648] HybridClock initialized: now 1786204335515186 us; error 45 us; skew 500 ppm
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:15.518056 9499 init.cc:377] Logged in from keytab as oryx/127.30.76.66@KRBTEST.COM (short username oryx)
I20260808 15:52:15.518940 9499 webserver.cc:460] Webserver started at http://127.30.76.66:42683/ using document root <none> and password file <none>
I20260808 15:52:15.519461 9499 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:15.519518 9499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:15.519799 9499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:15.521872 9499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "257d55a34238437184d068e0e146bd56"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "a973290a083104f3e0e99b65bff34527"
server_key_iv: "e8edb7742cff9d4891c989406c6dd47f"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.522554 9499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "257d55a34238437184d068e0e146bd56"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "a973290a083104f3e0e99b65bff34527"
server_key_iv: "e8edb7742cff9d4891c989406c6dd47f"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.526372 9499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 15:52:15.528898 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.530071 9499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:15.530200 9499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "257d55a34238437184d068e0e146bd56"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "a973290a083104f3e0e99b65bff34527"
server_key_iv: "e8edb7742cff9d4891c989406c6dd47f"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.530309 9499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:15.552987 9499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:15.555858 9499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:15.556066 9499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:15.557531 9499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:15.557607 9499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.557714 9499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:15.557765 9499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:15.568857 9499 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:39191
I20260808 15:52:15.568929 9698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:39191 every 8 connection(s)
I20260808 15:52:15.569906 9499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:15.571913 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.558479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:41891 (local address 127.30.76.126:42555)
0808 15:52:15.558610 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.558614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.559702 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 15:52:15.560773 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.560777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.560843 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.560910 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.562355 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.562965 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.564215 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.564508 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.564627 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.568075 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.568081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.568089 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.568115 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.569707 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.570324 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.570328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.570329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.570430 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.570866 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.570869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.570871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.571111 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.571201 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.571688 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.571795 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:52:15.572695 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.559011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.30.76.66:41891)
0808 15:52:15.559504 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.559592 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.560508 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.561331 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.561337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.561667 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:15.562139 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.562162 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.563100 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.563105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.564069 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.564078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.564687 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.564692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.564891 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.565501 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:15.565526 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:15.567898 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:15.569825 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.569832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.569847 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.570188 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.570543 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.570546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.570548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.570716 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.571214 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:15.571219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:15.571430 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.571624 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.571871 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260808 15:52:15.573966 9699 heartbeater.cc:347] Connected to a master server at 127.30.76.126:42555
I20260808 15:52:15.574146 9699 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:15.574616 9699 heartbeater.cc:511] Master 127.30.76.126:42555 requested a full tablet report, sending...
I20260808 15:52:15.575598 9212 ts_manager.cc:194] Registered new tserver with Master: 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:15.576210 9212 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:41891
I20260808 15:52:15.578326 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 9499
I20260808 15:52:15.578475 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:15.578781 31025 external_mini_cluster.cc:1428] Setting key 83590320221b2ed9cac3b14f95d96f0d
I20260808 15:52:15.583513 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.576274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:46909 (local address 127.30.76.126:42555)
0808 15:52:15.576509 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.576513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.576612 (+ 99us) server_negotiation.cc:407] Connection header received
0808 15:52:15.576723 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.576726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.576763 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.576852 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.577274 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.577776 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.578667 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.578931 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.579045 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.580207 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.580210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.580213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.580238 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.581789 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.582186 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.582192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.582197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.582288 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.582624 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.582627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.582629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.582824 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.582903 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.583230 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.583318 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260808 15:52:15.583587 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.576203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.30.76.66:46909)
0808 15:52:15.576529 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.576543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.576647 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.576871 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.576873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.577073 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:15.577163 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.577169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.577896 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.577899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.578533 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.578560 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.579049 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.579053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.579144 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.579675 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:15.579696 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:15.580072 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:15.581902 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.581904 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.581905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.582088 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.582398 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.582404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.582407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.582518 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.582922 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:15.582928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:15.583037 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.583159 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.583315 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":138,"threads_started":1}
WARNING: no policy specified for oryx/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab.
Entry for principal oryx/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab.
I20260808 15:52:15.607934 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:42555
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:15.717978 9711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:15.718200 9711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:15.718243 9711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:15.722309 9711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:15.722379 9711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:15.722445 9711 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:52:15.722525 9711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:15.727671 9711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.76.126:42555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:15.728618 9711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:15.729592 9711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:15.738976 9716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:15.739004 9719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:15.738976 9717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:15.739285 9711 server_base.cc:1034] running on GCE node
I20260808 15:52:15.739825 9711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:15.740439 9711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:15.741639 9711 hybrid_clock.cc:648] HybridClock initialized: now 1786204335741621 us; error 28 us; skew 500 ppm
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:15.744340 9711 init.cc:377] Logged in from keytab as oryx/127.30.76.67@KRBTEST.COM (short username oryx)
I20260808 15:52:15.745203 9711 webserver.cc:460] Webserver started at http://127.30.76.67:33817/ using document root <none> and password file <none>
I20260808 15:52:15.745762 9711 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:15.745857 9711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:15.746084 9711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:15.748149 9711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "32ae52a345af4ac0a5d71a528ef09401"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "5b3bf380fcaedadfb73869174b305ac1"
server_key_iv: "054b5d50d4d6fe9d259654a764f59afb"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.748704 9711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "32ae52a345af4ac0a5d71a528ef09401"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "5b3bf380fcaedadfb73869174b305ac1"
server_key_iv: "054b5d50d4d6fe9d259654a764f59afb"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.752362 9711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:15.754726 9728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.755961 9711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:15.756093 9711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "32ae52a345af4ac0a5d71a528ef09401"
format_stamp: "Formatted at 2026-08-08 15:52:15 on dist-test-slave-sb2z"
server_key: "5b3bf380fcaedadfb73869174b305ac1"
server_key_iv: "054b5d50d4d6fe9d259654a764f59afb"
server_key_version: "encryptionkey@0"
I20260808 15:52:15.756204 9711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:15.778240 9711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:15.781736 9711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:15.781951 9711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:15.783499 9711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:15.783563 9711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:15.783643 9711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:15.783818 9711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:15.796053 9711 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:37551
I20260808 15:52:15.796056 9892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:37551 every 8 connection(s)
I20260808 15:52:15.797030 9711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:15.799014 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.784662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:50365 (local address 127.30.76.126:42555)
0808 15:52:15.784803 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.784807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.785933 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 15:52:15.786989 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.786993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.787082 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.787191 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.788684 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.789739 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.791648 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.792105 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.792241 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.795165 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.795170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.795173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.795207 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.796880 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.797507 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.797513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.797518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.797625 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.798089 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.798091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.798093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.798270 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.798353 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.798806 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.798898 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:15.799978 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.785237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.30.76.67:50365)
0808 15:52:15.785781 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.785822 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.786727 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.787398 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.787408 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.787914 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:15.788515 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.788529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.789881 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.789887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.791424 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.791443 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.792295 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.792301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.792513 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.793178 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:15.793193 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:15.794990 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:15.797016 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.797022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.797034 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.797365 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.797764 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.797769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.797772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.797954 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.798370 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:15.798375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:15.798599 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.798756 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.799007 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":195,"threads_started":1}
I20260808 15:52:15.801194 9893 heartbeater.cc:347] Connected to a master server at 127.30.76.126:42555
I20260808 15:52:15.801331 9893 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:15.801869 9893 heartbeater.cc:511] Master 127.30.76.126:42555 requested a full tablet report, sending...
I20260808 15:52:15.802942 9211 ts_manager.cc:194] Registered new tserver with Master: 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551)
I20260808 15:52:15.803732 9211 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:50365
I20260808 15:52:15.804323 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 9711
I20260808 15:52:15.804464 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:15.804802 31025 external_mini_cluster.cc:1428] Setting key 7111d9aad684f0f59d12433d611a70eb
I20260808 15:52:15.806838 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:52:15.810735 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.803408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.30.76.67:33163)
0808 15:52:15.803704 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.803716 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.803825 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.804179 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.804182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.804365 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:15.804475 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.804482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.805421 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.805424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.806091 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.806100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.806616 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.806622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.806726 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.807202 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:15.807214 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:15.807506 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:15.809228 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.809232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.809234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.809434 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.809726 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:15.809728 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:15.809729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:15.809804 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:15.810261 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:15.810263 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:15.810336 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.810419 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.810516 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":119,"threads_started":1}
I20260808 15:52:15.810802 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.803479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:33163 (local address 127.30.76.126:42555)
0808 15:52:15.803610 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.803614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.803787 (+ 173us) server_negotiation.cc:407] Connection header received
0808 15:52:15.803933 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.803935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.803998 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.804070 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.804636 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.805292 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.806194 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.806489 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.806594 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.807706 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.807709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.807712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.807736 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.809091 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.809531 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.809536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.809539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.809632 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.809912 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.809915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.809918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.810161 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.810265 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.810508 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.810631 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:52:15.832701 9276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:15.826723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.30.76.126:42555)
0808 15:52:15.826892 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.826896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.826912 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:15.826949 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.826951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.826986 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.827072 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.827828 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.828345 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.829200 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.829421 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.829542 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.832564 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43656: BlockingRecv error: recv got EOF from 127.0.0.1:43656 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 15:52:15 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:15.841950 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.834414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.30.76.126:42555)
0808 15:52:15.834530 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.834533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.834700 (+ 167us) server_negotiation.cc:407] Connection header received
0808 15:52:15.834801 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.834803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.834837 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.834905 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:15.835428 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.835960 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.836720 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.836915 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.836988 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.839052 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:15.839063 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:15.839065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:15.839087 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:15.840338 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.840746 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.840750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.840752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.840832 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:15.841128 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:15.841131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:15.841133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:15.841301 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:15.841441 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.841684 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.841797 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:52:15.844574 9211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:15.847711 9211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:15.861763 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.855767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42179 (local address 127.0.0.1:57378)
0808 15:52:15.856310 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.856361 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.856482 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.857343 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.857351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.857375 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:15.857505 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.857511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.858759 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.858764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.860066 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.860078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.861200 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.861204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.861310 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.861347 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.861482 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.861615 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":75,"threads_started":1}
I20260808 15:52:15.861881 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.855900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.30.76.65:42179)
0808 15:52:15.856833 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.856841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.856865 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:52:15.856948 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.856955 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.857170 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.857680 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:15.857818 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.858637 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.860217 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.861069 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.861201 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.861348 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.861639 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.861731 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":69,"threads_started":1}
I20260808 15:52:15.862438 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.856468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:39191 (local address 127.0.0.1:59874)
0808 15:52:15.856890 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.856905 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.856985 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.857687 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.857690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.857708 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:15.857803 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.857811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.859201 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.859204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.860464 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.860474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.861790 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.861793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.862067 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.862103 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.862198 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.862297 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":176,"thread_start_us":102,"threads_started":1}
I20260808 15:52:15.862479 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.856646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.30.76.66:39191)
0808 15:52:15.857254 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.857268 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.857291 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:52:15.857365 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.857370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.857534 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.857712 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:15.857935 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.859076 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.860645 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.861464 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.861568 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.861722 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.862248 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.862347 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":109,"threads_started":1}
I20260808 15:52:15.862651 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.856386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.30.76.67:37551)
0808 15:52:15.857302 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:15.857315 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:15.857335 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:52:15.857404 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:15.857408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:15.857600 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:15.857729 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:15.858096 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.859186 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.860631 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.861459 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.861568 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.861722 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:15.862376 (+ 654us) server_negotiation.cc:299] Negotiation successful
0808 15:52:15.862480 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":120,"threads_started":1}
I20260808 15:52:15.862903 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:15.856020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:37551 (local address 127.0.0.1:56180)
0808 15:52:15.856916 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:15.856931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:15.857025 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:15.857862 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:15.857867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:15.857884 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:15.857968 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.857973 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.859300 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.859304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.860467 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:15.860477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:15.861790 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:15.861794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:15.862113 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:15.862136 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:52:15.862200 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 15:52:15.862297 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":59,"thread_start_us":122,"threads_started":1}
I20260808 15:52:15.865000 9399 tablet_service.cc:1467] Processing CreateTablet for tablet 91201027b70246a9963ce1e254b60d74 (DEFAULT_TABLE table=test-table [id=ca0ea1173c014408863410510f0c42fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:15.865278 9806 tablet_service.cc:1467] Processing CreateTablet for tablet 91201027b70246a9963ce1e254b60d74 (DEFAULT_TABLE table=test-table [id=ca0ea1173c014408863410510f0c42fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:15.865677 9596 tablet_service.cc:1467] Processing CreateTablet for tablet 91201027b70246a9963ce1e254b60d74 (DEFAULT_TABLE table=test-table [id=ca0ea1173c014408863410510f0c42fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:15.866158 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91201027b70246a9963ce1e254b60d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:15.866227 9806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91201027b70246a9963ce1e254b60d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:15.866554 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91201027b70246a9963ce1e254b60d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:15.872625 9935 tablet_bootstrap.cc:492] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Bootstrap starting.
I20260808 15:52:15.874002 9939 tablet_bootstrap.cc:492] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Bootstrap starting.
I20260808 15:52:15.874037 9937 tablet_bootstrap.cc:492] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Bootstrap starting.
I20260808 15:52:15.874914 9935 tablet_bootstrap.cc:654] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:15.875713 9935 log.cc:824] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:15.876075 9937 tablet_bootstrap.cc:654] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:15.876075 9939 tablet_bootstrap.cc:654] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:15.876871 9937 log.cc:824] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:15.876871 9939 log.cc:824] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:15.877916 9935 tablet_bootstrap.cc:492] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: No bootstrap required, opened a new log
I20260808 15:52:15.878115 9935 ts_tablet_manager.cc:1397] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:52:15.878494 9939 tablet_bootstrap.cc:492] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: No bootstrap required, opened a new log
I20260808 15:52:15.878705 9939 ts_tablet_manager.cc:1397] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 15:52:15.880820 9937 tablet_bootstrap.cc:492] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: No bootstrap required, opened a new log
I20260808 15:52:15.881021 9937 ts_tablet_manager.cc:1397] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 15:52:15.881207 9935 raft_consensus.cc:348] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:15.881433 9935 raft_consensus.cc:374] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:15.881495 9935 raft_consensus.cc:729] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Initialized, Role: FOLLOWER
I20260808 15:52:15.881953 9935 consensus_queue.cc:260] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:15.882337 9939 raft_consensus.cc:348] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:15.882560 9939 raft_consensus.cc:374] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:15.882611 9939 raft_consensus.cc:729] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257d55a34238437184d068e0e146bd56, State: Initialized, Role: FOLLOWER
I20260808 15:52:15.882954 9935 ts_tablet_manager.cc:1428] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260808 15:52:15.883013 9893 heartbeater.cc:503] Master 127.30.76.126:42555 was elected leader, sending a full tablet report...
I20260808 15:52:15.883065 9939 consensus_queue.cc:260] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:15.883715 9699 heartbeater.cc:503] Master 127.30.76.126:42555 was elected leader, sending a full tablet report...
I20260808 15:52:15.884022 9939 ts_tablet_manager.cc:1428] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 15:52:15.884975 9937 raft_consensus.cc:348] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:15.885187 9937 raft_consensus.cc:374] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:15.885248 9937 raft_consensus.cc:729] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a9269d2c6044668385fd7f0081b3b7, State: Initialized, Role: FOLLOWER
I20260808 15:52:15.885636 9937 consensus_queue.cc:260] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:15.886171 9489 heartbeater.cc:503] Master 127.30.76.126:42555 was elected leader, sending a full tablet report...
I20260808 15:52:15.887303 9937 ts_tablet_manager.cc:1428] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:52:16.014323 9944 raft_consensus.cc:482] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:16.014518 9944 raft_consensus.cc:504] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:16.015654 9944 leader_election.cc:288] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:16.019205 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.015876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42179 (local address 127.30.76.67:60799)
0808 15:52:16.016027 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:16.016044 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:16.016152 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:16.016444 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:16.016447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:16.016466 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:16.016578 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.016583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.017313 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.017316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.018088 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.018097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.018766 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.018769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.018871 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.018899 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:52:16.018986 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 15:52:16.019063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 15:52:16.019205 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.015995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:60799 (local address 127.30.76.65:42179)
0808 15:52:16.016152 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.016156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.016176 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:52:16.016243 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.016246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.016306 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.016391 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:16.016695 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.017193 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.018225 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.018641 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.018773 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.018825 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.019016 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.019083 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:52:16.019902 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91201027b70246a9963ce1e254b60d74" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7" is_pre_election: true
I20260808 15:52:16.020151 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.016151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:39191 (local address 127.30.76.67:46961)
0808 15:52:16.016489 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:16.016503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:16.016637 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:16.016907 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:16.016910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:16.016926 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:16.017058 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.017063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.017802 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.017807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.018938 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.018948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.019797 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.019800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.019870 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.019888 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 15:52:16.019950 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 15:52:16.020000 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":118,"threads_started":1}
I20260808 15:52:16.020187 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.016349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:46961 (local address 127.30.76.66:39191)
0808 15:52:16.016483 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.016488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.016587 (+ 99us) server_negotiation.cc:407] Connection header received
0808 15:52:16.016757 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.016760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.016810 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.016894 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:16.017192 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.017689 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.019069 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.019663 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.019785 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.019827 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.019992 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.020062 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:52:16.020270 9417 raft_consensus.cc:2454] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:16.020689 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91201027b70246a9963ce1e254b60d74" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56" is_pre_election: true
I20260808 15:52:16.020744 9730 leader_election.cc:302] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ae52a345af4ac0a5d71a528ef09401, e8a9269d2c6044668385fd7f0081b3b7; no voters:
I20260808 15:52:16.020975 9616 raft_consensus.cc:2454] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:16.021021 9944 raft_consensus.cc:2781] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:16.021132 9944 raft_consensus.cc:482] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:16.021209 9944 raft_consensus.cc:3037] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.022238 9944 raft_consensus.cc:504] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:16.022567 9944 leader_election.cc:288] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Requested vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:16.022917 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91201027b70246a9963ce1e254b60d74" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7"
I20260808 15:52:16.023018 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91201027b70246a9963ce1e254b60d74" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56"
I20260808 15:52:16.023041 9417 raft_consensus.cc:3037] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.023140 9616 raft_consensus.cc:3037] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.024477 9417 raft_consensus.cc:2454] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:16.024477 9616 raft_consensus.cc:2454] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:16.024794 9730 leader_election.cc:302] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d55a34238437184d068e0e146bd56, 32ae52a345af4ac0a5d71a528ef09401; no voters:
I20260808 15:52:16.024966 9944 raft_consensus.cc:2781] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:16.025187 9944 raft_consensus.cc:686] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Becoming Leader. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Running, Role: LEADER
I20260808 15:52:16.025565 9944 consensus_queue.cc:237] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } }
I20260808 15:52:16.028803 9212 catalog_manager.cc:5495] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } health_report { overall_health: UNKNOWN } } }
W20260808 15:52:16.048064 9896 tablet.cc:2369] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:16.054471 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.050981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.30.76.67:37551)
0808 15:52:16.051123 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.051137 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.051152 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:16.051267 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.051269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.051320 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.051429 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:16.051694 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.052193 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.053062 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.053290 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.053402 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.053693 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:16.054086 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:16.054221 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.054281 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.054333 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:52:16.059930 9616 raft_consensus.cc:1261] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:16.060034 9417 raft_consensus.cc:1261] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:16.060668 9953 consensus_queue.cc:1035] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:16.060807 9944 consensus_queue.cc:1035] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:16.072489 9958 mvcc.cc:205] Tried to move back new op lower bound from 7316292960494485504 to 7316292960363499520. Current Snapshot: MvccSnapshot[applied={T|T < 7316292960494485504}]
I20260808 15:52:16.080736 9210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:16.080894 9210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:16.083348 9210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Sending DeleteTablet for 3 replicas of tablet 91201027b70246a9963ce1e254b60d74
I20260808 15:52:16.084125 9806 tablet_service.cc:1514] Processing DeleteTablet for tablet 91201027b70246a9963ce1e254b60d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:16 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56180
I20260808 15:52:16.084210 9399 tablet_service.cc:1514] Processing DeleteTablet for tablet 91201027b70246a9963ce1e254b60d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:16 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:57378
I20260808 15:52:16.084568 9596 tablet_service.cc:1514] Processing DeleteTablet for tablet 91201027b70246a9963ce1e254b60d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:16 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59874
I20260808 15:52:16.084928 9973 tablet_replica.cc:330] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: stopping tablet replica
I20260808 15:52:16.085000 9210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:52:16.085210 9973 raft_consensus.cc:2229] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:16.085372 9971 tablet_replica.cc:330] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: stopping tablet replica
W20260808 15:52:16.085398 9210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:16.085491 9973 raft_consensus.cc:2258] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:16.085634 9971 raft_consensus.cc:2229] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:16.086024 9971 raft_consensus.cc:2258] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:16.086311 9972 tablet_replica.cc:330] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: stopping tablet replica
I20260808 15:52:16.086581 9972 raft_consensus.cc:2229] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:16.086859 9972 raft_consensus.cc:2258] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:16.087627 9971 ts_tablet_manager.cc:1905] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:16.088189 9972 ts_tablet_manager.cc:1905] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:16.090487 9971 ts_tablet_manager.cc:1918] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:16.090586 9971 log.cc:1196] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/91201027b70246a9963ce1e254b60d74
I20260808 15:52:16.090903 9971 ts_tablet_manager.cc:1939] T 91201027b70246a9963ce1e254b60d74 P 32ae52a345af4ac0a5d71a528ef09401: Deleting consensus metadata
I20260808 15:52:16.090958 9806 tablet_service.cc:1467] Processing CreateTablet for tablet ef13804287374bd285f91e9f47763905 (DEFAULT_TABLE table=test-table [id=9783d2f41ca14f619a9c05ac734a6d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:16.091190 9806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef13804287374bd285f91e9f47763905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.091238 9399 tablet_service.cc:1467] Processing CreateTablet for tablet ef13804287374bd285f91e9f47763905 (DEFAULT_TABLE table=test-table [id=9783d2f41ca14f619a9c05ac734a6d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:16.091481 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef13804287374bd285f91e9f47763905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.093017 9935 tablet_bootstrap.cc:492] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Bootstrap starting.
I20260808 15:52:16.093204 9596 tablet_service.cc:1467] Processing CreateTablet for tablet ef13804287374bd285f91e9f47763905 (DEFAULT_TABLE table=test-table [id=9783d2f41ca14f619a9c05ac734a6d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:16.093457 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef13804287374bd285f91e9f47763905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.094043 9935 tablet_bootstrap.cc:654] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.095782 9937 tablet_bootstrap.cc:492] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Bootstrap starting.
I20260808 15:52:16.096294 9973 ts_tablet_manager.cc:1905] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:16.096725 9937 tablet_bootstrap.cc:654] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.097304 9939 tablet_bootstrap.cc:492] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Bootstrap starting.
I20260808 15:52:16.098224 9939 tablet_bootstrap.cc:654] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.098798 9973 ts_tablet_manager.cc:1918] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:16.098932 9973 log.cc:1196] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/91201027b70246a9963ce1e254b60d74
I20260808 15:52:16.098987 9192 catalog_manager.cc:4841] TS 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551): tablet 91201027b70246a9963ce1e254b60d74 (table test-table [id=ca0ea1173c014408863410510f0c42fe]) successfully deleted
I20260808 15:52:16.099275 9973 ts_tablet_manager.cc:1939] T 91201027b70246a9963ce1e254b60d74 P 257d55a34238437184d068e0e146bd56: Deleting consensus metadata
I20260808 15:52:16.099671 9972 ts_tablet_manager.cc:1918] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:16.099766 9972 log.cc:1196] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/91201027b70246a9963ce1e254b60d74
I20260808 15:52:16.100106 9972 ts_tablet_manager.cc:1939] T 91201027b70246a9963ce1e254b60d74 P e8a9269d2c6044668385fd7f0081b3b7: Deleting consensus metadata
I20260808 15:52:16.100405 9939 tablet_bootstrap.cc:492] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: No bootstrap required, opened a new log
I20260808 15:52:16.100476 9939 ts_tablet_manager.cc:1397] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:16.100915 9939 raft_consensus.cc:348] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.101032 9939 raft_consensus.cc:374] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.101063 9939 raft_consensus.cc:729] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257d55a34238437184d068e0e146bd56, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.101181 9939 consensus_queue.cc:260] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.101218 9190 catalog_manager.cc:4841] TS e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179): tablet 91201027b70246a9963ce1e254b60d74 (table test-table [id=ca0ea1173c014408863410510f0c42fe]) successfully deleted
I20260808 15:52:16.101410 9939 ts_tablet_manager.cc:1428] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:16.101862 9937 tablet_bootstrap.cc:492] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: No bootstrap required, opened a new log
I20260808 15:52:16.101951 9937 ts_tablet_manager.cc:1397] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 15:52:16.102144 9935 tablet_bootstrap.cc:492] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: No bootstrap required, opened a new log
I20260808 15:52:16.102231 9935 ts_tablet_manager.cc:1397] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260808 15:52:16.102550 9937 raft_consensus.cc:348] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.102658 9937 raft_consensus.cc:374] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.102692 9937 raft_consensus.cc:729] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a9269d2c6044668385fd7f0081b3b7, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.102636 9935 raft_consensus.cc:348] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.102739 9935 raft_consensus.cc:374] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.102774 9935 raft_consensus.cc:729] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.102813 9937 consensus_queue.cc:260] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.102880 9935 consensus_queue.cc:260] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.103057 9937 ts_tablet_manager.cc:1428] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:16.103137 9935 ts_tablet_manager.cc:1428] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 15:52:16.103273 9190 catalog_manager.cc:4841] TS 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191): tablet 91201027b70246a9963ce1e254b60d74 (table test-table [id=ca0ea1173c014408863410510f0c42fe]) successfully deleted
I20260808 15:52:16.264971 9944 raft_consensus.cc:482] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:16.265158 9944 raft_consensus.cc:504] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.265656 9944 leader_election.cc:288] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:16.266204 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef13804287374bd285f91e9f47763905" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7" is_pre_election: true
I20260808 15:52:16.266270 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef13804287374bd285f91e9f47763905" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56" is_pre_election: true
I20260808 15:52:16.266379 9417 raft_consensus.cc:2454] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:16.266461 9616 raft_consensus.cc:2454] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:16.266741 9730 leader_election.cc:302] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ae52a345af4ac0a5d71a528ef09401, e8a9269d2c6044668385fd7f0081b3b7; no voters:
I20260808 15:52:16.266937 9944 raft_consensus.cc:2781] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:16.266999 9944 raft_consensus.cc:482] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:16.267040 9944 raft_consensus.cc:3037] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.268085 9944 raft_consensus.cc:504] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.268405 9944 leader_election.cc:288] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Requested vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:16.268756 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef13804287374bd285f91e9f47763905" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7"
I20260808 15:52:16.268795 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef13804287374bd285f91e9f47763905" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56"
I20260808 15:52:16.268857 9417 raft_consensus.cc:3037] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.268927 9616 raft_consensus.cc:3037] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.269842 9616 raft_consensus.cc:2454] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:16.270061 9417 raft_consensus.cc:2454] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:16.270141 9730 leader_election.cc:302] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d55a34238437184d068e0e146bd56, 32ae52a345af4ac0a5d71a528ef09401; no voters:
I20260808 15:52:16.270421 9944 raft_consensus.cc:2781] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:16.270525 9944 raft_consensus.cc:686] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Becoming Leader. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Running, Role: LEADER
I20260808 15:52:16.270677 9944 consensus_queue.cc:237] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.272523 9211 catalog_manager.cc:5495] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } health_report { overall_health: HEALTHY } } }
Aug 08 15:52:16 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204335, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.126@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:16.308959 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.301160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.30.76.126:42555)
0808 15:52:16.301294 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.301299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.301489 (+ 190us) server_negotiation.cc:407] Connection header received
0808 15:52:16.301584 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.301586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.301625 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.301721 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:16.302410 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.302916 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.303776 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.303983 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.304063 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.306209 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:16.306213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:16.306214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:16.306235 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:16.307436 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:16.307912 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:16.307915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:16.307916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:16.307990 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:16.308232 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:16.308234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:16.308235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:16.308390 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:16.308465 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.308744 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.308845 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:52:16.309190 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.301123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:42555 (local address 127.0.0.1:43672)
0808 15:52:16.301403 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:16.301416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:16.301511 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:16.301750 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:16.301752 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:16.302216 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:16.302308 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.302314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.303006 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.303008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.303675 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.303684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.304077 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.304079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.304141 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.304724 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:16.304736 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:16.306085 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:16.307577 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:16.307585 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:16.307590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:16.307826 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:16.308071 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:16.308073 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:16.308074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:16.308155 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:16.308477 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:16.308482 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:16.308578 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 15:52:16.308708 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 15:52:16.308945 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":55,"thread_start_us":110,"threads_started":1}
I20260808 15:52:16.312917 9211 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:43672:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 15:52:16.313666 9211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:16.318392 9399 tablet_service.cc:1467] Processing CreateTablet for tablet 2aca54898bad46208fa9da77bdea5054 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fd4281360864dce8eb752a96d66f5ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:52:16.318460 9596 tablet_service.cc:1467] Processing CreateTablet for tablet 2aca54898bad46208fa9da77bdea5054 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fd4281360864dce8eb752a96d66f5ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:52:16.318514 9806 tablet_service.cc:1467] Processing CreateTablet for tablet 2aca54898bad46208fa9da77bdea5054 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fd4281360864dce8eb752a96d66f5ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:52:16.318833 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aca54898bad46208fa9da77bdea5054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.318856 9806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aca54898bad46208fa9da77bdea5054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.318856 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aca54898bad46208fa9da77bdea5054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.321468 9937 tablet_bootstrap.cc:492] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: Bootstrap starting.
I20260808 15:52:16.321609 9935 tablet_bootstrap.cc:492] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: Bootstrap starting.
I20260808 15:52:16.321671 9939 tablet_bootstrap.cc:492] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: Bootstrap starting.
I20260808 15:52:16.322451 9937 tablet_bootstrap.cc:654] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.322544 9939 tablet_bootstrap.cc:654] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.322544 9935 tablet_bootstrap.cc:654] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: Neither blocks nor log segments found. Creating new log.
W20260808 15:52:16.323122 9702 tablet.cc:2369] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:16.323889 9935 tablet_bootstrap.cc:492] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: No bootstrap required, opened a new log
I20260808 15:52:16.323975 9939 tablet_bootstrap.cc:492] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: No bootstrap required, opened a new log
I20260808 15:52:16.323988 9935 ts_tablet_manager.cc:1397] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 15:52:16.324062 9939 ts_tablet_manager.cc:1397] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:16.324296 9937 tablet_bootstrap.cc:492] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: No bootstrap required, opened a new log
I20260808 15:52:16.324388 9937 ts_tablet_manager.cc:1397] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 15:52:16.324445 9935 raft_consensus.cc:348] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.324601 9935 raft_consensus.cc:374] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.324652 9935 raft_consensus.cc:729] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.324548 9939 raft_consensus.cc:348] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.324733 9939 raft_consensus.cc:374] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.324777 9939 raft_consensus.cc:729] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257d55a34238437184d068e0e146bd56, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.324780 9935 consensus_queue.cc:260] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.324893 9937 raft_consensus.cc:348] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.324914 9939 consensus_queue.cc:260] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.324998 9937 raft_consensus.cc:374] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.325034 9937 raft_consensus.cc:729] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a9269d2c6044668385fd7f0081b3b7, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.325026 9944 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.325176 9944 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.325165 9937 consensus_queue.cc:260] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.325472 9937 ts_tablet_manager.cc:1428] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:16.325781 9946 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.325922 9946 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.325415 9949 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.326433 9949 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.325098 9935 ts_tablet_manager.cc:1428] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:16.326793 9939 ts_tablet_manager.cc:1428] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260808 15:52:16.345715 9491 tablet.cc:2369] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:52:16.573786 9944 raft_consensus.cc:482] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:16.573935 9944 raft_consensus.cc:504] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.574422 9944 leader_election.cc:288] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:16.574905 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aca54898bad46208fa9da77bdea5054" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56" is_pre_election: true
I20260808 15:52:16.574916 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aca54898bad46208fa9da77bdea5054" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7" is_pre_election: true
I20260808 15:52:16.575038 9616 raft_consensus.cc:2454] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:16.575105 9417 raft_consensus.cc:2454] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:16.575311 9730 leader_election.cc:302] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d55a34238437184d068e0e146bd56, 32ae52a345af4ac0a5d71a528ef09401; no voters:
I20260808 15:52:16.575572 9944 raft_consensus.cc:2781] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:16.575642 9944 raft_consensus.cc:482] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:16.575698 9944 raft_consensus.cc:3037] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.576663 9944 raft_consensus.cc:504] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.577023 9944 leader_election.cc:288] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Requested vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:16.577364 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aca54898bad46208fa9da77bdea5054" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7"
I20260808 15:52:16.577430 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aca54898bad46208fa9da77bdea5054" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56"
I20260808 15:52:16.577507 9417 raft_consensus.cc:3037] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.577541 9616 raft_consensus.cc:3037] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:16.578373 9417 raft_consensus.cc:2454] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:16.578586 9616 raft_consensus.cc:2454] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:16.578668 9730 leader_election.cc:302] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ae52a345af4ac0a5d71a528ef09401, e8a9269d2c6044668385fd7f0081b3b7; no voters:
I20260808 15:52:16.578835 9944 raft_consensus.cc:2781] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:16.578919 9944 raft_consensus.cc:686] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Becoming Leader. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Running, Role: LEADER
I20260808 15:52:16.579048 9944 consensus_queue.cc:237] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.579664 9963 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: TxnStatusTablet state changed. Reason: New leader 32ae52a345af4ac0a5d71a528ef09401. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.579775 9963 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:52:16.579982 9996 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 15:52:16.580171 9996 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 15:52:16.581142 9212 catalog_manager.cc:5495] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } health_report { overall_health: HEALTHY } } }
I20260808 15:52:16.620865 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.617475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:37551 (local address 127.0.0.1:56202)
0808 15:52:16.617630 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:16.617655 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:16.617784 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:16.618112 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:16.618115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:16.618123 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:52:16.618233 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.618238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.618972 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.618975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.619631 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.619647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.620112 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.620116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.620210 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.620253 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:52:16.620563 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:52:16.620580 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:52:16.620660 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 15:52:16.620741 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 15:52:16.620870 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.617616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.30.76.67:37551)
0808 15:52:16.617877 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.617883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.617898 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:16.617951 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.617953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.617996 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.618089 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:16.618343 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.618859 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.619751 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.619999 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.620110 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.620392 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:16.620475 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:16.620558 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.620673 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.620722 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":66,"threads_started":1}
I20260808 15:52:16.624022 9417 raft_consensus.cc:1261] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:16.624053 9616 raft_consensus.cc:1261] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:16.624462 9944 consensus_queue.cc:1035] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:16.624605 9963 consensus_queue.cc:1035] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:16.625945 9949 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: TxnStatusTablet state changed. Reason: New leader 32ae52a345af4ac0a5d71a528ef09401. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.626060 9949 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.626636 9946 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: TxnStatusTablet state changed. Reason: New leader 32ae52a345af4ac0a5d71a528ef09401. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.626757 9946 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.629040 9963 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.629156 9963 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:52:16.629251 9963 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.629321 9963 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 32ae52a345af4ac0a5d71a528ef09401: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:52:16.629527 9949 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.629622 9949 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P e8a9269d2c6044668385fd7f0081b3b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.630414 9946 tablet_replica.cc:439] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } } }
I20260808 15:52:16.630515 9946 tablet_replica.cc:442] T 2aca54898bad46208fa9da77bdea5054 P 257d55a34238437184d068e0e146bd56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:52:16.632616 10002 mvcc.cc:205] Tried to move back new op lower bound from 7316292962807480320 to 7316292962628927488. Current Snapshot: MvccSnapshot[applied={T|T < 7316292962807480320 or (T in {7316292962807480320})}]
I20260808 15:52:16.632833 10003 mvcc.cc:205] Tried to move back new op lower bound from 7316292962807480320 to 7316292962628927488. Current Snapshot: MvccSnapshot[applied={T|T < 7316292962807480320 or (T in {7316292962807480320})}]
I20260808 15:52:16.644019 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.640369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:37551 (local address 127.30.76.67:36309)
0808 15:52:16.640728 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:16.640740 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:16.640826 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:16.641055 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:16.641057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:16.641063 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:52:16.641170 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.641175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.642148 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.642150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.642737 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:16.642744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.643292 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.643295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:16.643381 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.643422 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:52:16.643733 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:52:16.643739 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:52:16.643792 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 15:52:16.643869 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260808 15:52:16.644034 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.640466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:36309 (local address 127.30.76.67:37551)
0808 15:52:16.640588 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.640594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.640810 (+ 216us) server_negotiation.cc:407] Connection header received
0808 15:52:16.640892 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.640895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.640961 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.641042 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:16.641266 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.642036 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.642849 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.643183 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.643286 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.643560 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:16.643647 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:16.643721 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.643804 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.643880 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":43}
I20260808 15:52:16.650249 9616 raft_consensus.cc:1261] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:16.650312 9417 raft_consensus.cc:1261] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:16.650728 10007 consensus_queue.cc:1035] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:16.650866 10011 consensus_queue.cc:1035] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:16.688671 9211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:16.688899 9211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:16.691012 9211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Sending DeleteTablet for 3 replicas of tablet ef13804287374bd285f91e9f47763905
I20260808 15:52:16.691795 9596 tablet_service.cc:1514] Processing DeleteTablet for tablet ef13804287374bd285f91e9f47763905 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:16 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59874
I20260808 15:52:16.691812 9399 tablet_service.cc:1514] Processing DeleteTablet for tablet ef13804287374bd285f91e9f47763905 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:16 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:57378
I20260808 15:52:16.692168 9805 tablet_service.cc:1514] Processing DeleteTablet for tablet ef13804287374bd285f91e9f47763905 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:16 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56180
I20260808 15:52:16.692528 10026 tablet_replica.cc:330] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: stopping tablet replica
I20260808 15:52:16.692659 10026 raft_consensus.cc:2229] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:16.692853 10026 raft_consensus.cc:2258] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:16.693586 10026 ts_tablet_manager.cc:1905] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:16.696362 10024 tablet_replica.cc:330] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: stopping tablet replica
I20260808 15:52:16.696503 10024 raft_consensus.cc:2229] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:16.696624 10024 raft_consensus.cc:2258] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:16.696905 10025 tablet_replica.cc:330] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: stopping tablet replica
I20260808 15:52:16.697052 10025 raft_consensus.cc:2229] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:16.697098 10026 ts_tablet_manager.cc:1918] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:52:16.697186 10026 log.cc:1196] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/ef13804287374bd285f91e9f47763905
I20260808 15:52:16.697190 10025 raft_consensus.cc:2258] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:16.697486 10026 ts_tablet_manager.cc:1939] T ef13804287374bd285f91e9f47763905 P 32ae52a345af4ac0a5d71a528ef09401: Deleting consensus metadata
I20260808 15:52:16.697746 10024 ts_tablet_manager.cc:1905] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:16.697902 10025 ts_tablet_manager.cc:1905] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:16.698238 9192 catalog_manager.cc:4841] TS 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551): tablet ef13804287374bd285f91e9f47763905 (table test-table [id=9783d2f41ca14f619a9c05ac734a6d1e]) successfully deleted
I20260808 15:52:16.701229 10025 ts_tablet_manager.cc:1918] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:52:16.701277 10024 ts_tablet_manager.cc:1918] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:52:16.701330 10025 log.cc:1196] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/ef13804287374bd285f91e9f47763905
I20260808 15:52:16.701359 10024 log.cc:1196] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/ef13804287374bd285f91e9f47763905
I20260808 15:52:16.701586 10025 ts_tablet_manager.cc:1939] T ef13804287374bd285f91e9f47763905 P 257d55a34238437184d068e0e146bd56: Deleting consensus metadata
I20260808 15:52:16.701679 10024 ts_tablet_manager.cc:1939] T ef13804287374bd285f91e9f47763905 P e8a9269d2c6044668385fd7f0081b3b7: Deleting consensus metadata
I20260808 15:52:16.702374 9190 catalog_manager.cc:4841] TS 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191): tablet ef13804287374bd285f91e9f47763905 (table test-table [id=9783d2f41ca14f619a9c05ac734a6d1e]) successfully deleted
I20260808 15:52:16.702554 9190 catalog_manager.cc:4841] TS e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179): tablet ef13804287374bd285f91e9f47763905 (table test-table [id=9783d2f41ca14f619a9c05ac734a6d1e]) successfully deleted
Aug 08 15:52:16 dist-test-slave-sb2z krb5kdc[9145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:52:16 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:16 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:52:16.726377 9276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:16.719158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.30.76.126:42555)
0808 15:52:16.719302 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.719307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.719440 (+ 133us) server_negotiation.cc:407] Connection header received
0808 15:52:16.719648 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.719651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.719702 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.719790 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:16.720634 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.721450 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.722344 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.722715 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.722822 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.726269 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43678: BlockingRecv error: recv got EOF from 127.0.0.1:43678 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 15:52:16 dist-test-slave-sb2z krb5kdc[9145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM
I20260808 15:52:16.737322 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:16.728549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.30.76.126:42555)
0808 15:52:16.728688 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:16.728692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:16.728896 (+ 204us) server_negotiation.cc:407] Connection header received
0808 15:52:16.729017 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:16.729020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:16.729062 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:16.729153 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:16.729740 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.730242 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.731022 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:16.731262 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:16.731373 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:16.733961 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:16.733964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:16.733966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:16.733990 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:16.735506 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:16.736017 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:16.736020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:16.736021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:16.736098 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:16.736376 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:16.736378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:16.736379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:16.736573 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:16.736717 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:16.737012 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 15:52:16.737149 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:52:16.739989 9211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:16.740468 9211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:16.745255 9596 tablet_service.cc:1467] Processing CreateTablet for tablet 44bfba5c4c024c44b6ea9ef34decb991 (DEFAULT_TABLE table=test-table [id=f61d81965dcb4999bd55e15b87bd325e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:16.745255 9399 tablet_service.cc:1467] Processing CreateTablet for tablet 44bfba5c4c024c44b6ea9ef34decb991 (DEFAULT_TABLE table=test-table [id=f61d81965dcb4999bd55e15b87bd325e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:16.745366 9805 tablet_service.cc:1467] Processing CreateTablet for tablet 44bfba5c4c024c44b6ea9ef34decb991 (DEFAULT_TABLE table=test-table [id=f61d81965dcb4999bd55e15b87bd325e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:16.745575 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44bfba5c4c024c44b6ea9ef34decb991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.745575 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44bfba5c4c024c44b6ea9ef34decb991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.745734 9805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44bfba5c4c024c44b6ea9ef34decb991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:16.747797 9935 tablet_bootstrap.cc:492] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Bootstrap starting.
I20260808 15:52:16.748062 9937 tablet_bootstrap.cc:492] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Bootstrap starting.
I20260808 15:52:16.748695 9935 tablet_bootstrap.cc:654] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.748980 9937 tablet_bootstrap.cc:654] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.749248 9939 tablet_bootstrap.cc:492] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Bootstrap starting.
I20260808 15:52:16.749891 9935 tablet_bootstrap.cc:492] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: No bootstrap required, opened a new log
I20260808 15:52:16.749979 9935 ts_tablet_manager.cc:1397] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:16.750221 9939 tablet_bootstrap.cc:654] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:16.750542 9935 raft_consensus.cc:348] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.750650 9935 raft_consensus.cc:374] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.750667 9937 tablet_bootstrap.cc:492] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: No bootstrap required, opened a new log
I20260808 15:52:16.750692 9935 raft_consensus.cc:729] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.750756 9937 ts_tablet_manager.cc:1397] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:16.750851 9935 consensus_queue.cc:260] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.751144 9935 ts_tablet_manager.cc:1428] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 15:52:16.751255 9937 raft_consensus.cc:348] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.751374 9937 raft_consensus.cc:374] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.751407 9937 raft_consensus.cc:729] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a9269d2c6044668385fd7f0081b3b7, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.751562 9937 consensus_queue.cc:260] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.751937 9937 ts_tablet_manager.cc:1428] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:16.752715 9939 tablet_bootstrap.cc:492] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: No bootstrap required, opened a new log
I20260808 15:52:16.752808 9939 ts_tablet_manager.cc:1397] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 15:52:16.753345 9939 raft_consensus.cc:348] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.753491 9939 raft_consensus.cc:374] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:16.753542 9939 raft_consensus.cc:729] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257d55a34238437184d068e0e146bd56, State: Initialized, Role: FOLLOWER
I20260808 15:52:16.753674 9939 consensus_queue.cc:260] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:16.754005 9939 ts_tablet_manager.cc:1428] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:17.111594 10011 raft_consensus.cc:482] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:17.111752 10011 raft_consensus.cc:504] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.112200 10011 leader_election.cc:288] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:17.112573 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bfba5c4c024c44b6ea9ef34decb991" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7" is_pre_election: true
I20260808 15:52:17.112653 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bfba5c4c024c44b6ea9ef34decb991" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56" is_pre_election: true
I20260808 15:52:17.112767 9417 raft_consensus.cc:2454] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:17.112818 9616 raft_consensus.cc:2454] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 0.
I20260808 15:52:17.113121 9730 leader_election.cc:302] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ae52a345af4ac0a5d71a528ef09401, e8a9269d2c6044668385fd7f0081b3b7; no voters:
I20260808 15:52:17.113312 10011 raft_consensus.cc:2781] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:17.113386 10011 raft_consensus.cc:482] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:17.113440 10011 raft_consensus.cc:3037] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:17.114468 10011 raft_consensus.cc:504] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.114776 10011 leader_election.cc:288] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Requested vote from peers e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179), 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191)
I20260808 15:52:17.115120 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bfba5c4c024c44b6ea9ef34decb991" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a9269d2c6044668385fd7f0081b3b7"
I20260808 15:52:17.115157 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bfba5c4c024c44b6ea9ef34decb991" candidate_uuid: "32ae52a345af4ac0a5d71a528ef09401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56"
I20260808 15:52:17.115252 9417 raft_consensus.cc:3037] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:17.115270 9616 raft_consensus.cc:3037] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:17.116180 9616 raft_consensus.cc:2454] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:17.116254 9417 raft_consensus.cc:2454] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae52a345af4ac0a5d71a528ef09401 in term 1.
I20260808 15:52:17.116460 9730 leader_election.cc:302] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d55a34238437184d068e0e146bd56, 32ae52a345af4ac0a5d71a528ef09401; no voters:
I20260808 15:52:17.116640 10011 raft_consensus.cc:2781] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:17.116751 10011 raft_consensus.cc:686] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Becoming Leader. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Running, Role: LEADER
I20260808 15:52:17.116912 10011 consensus_queue.cc:237] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.118459 9212 catalog_manager.cc:5495] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "32ae52a345af4ac0a5d71a528ef09401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } health_report { overall_health: HEALTHY } } }
I20260808 15:52:17.204473 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.201192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.30.76.67:37551)
0808 15:52:17.201427 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:17.201431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:17.201461 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:52:17.201506 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:17.201508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:17.201555 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:17.201641 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:17.201868 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.202510 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.203454 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.203718 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.203803 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.204048 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:17.204138 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:17.204195 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:17.204297 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 15:52:17.204347 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260808 15:52:17.206702 9417 raft_consensus.cc:1261] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:17.206710 9616 raft_consensus.cc:1261] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Refusing update from remote peer 32ae52a345af4ac0a5d71a528ef09401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:17.207149 10011 consensus_queue.cc:1035] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:17.207304 9944 consensus_queue.cc:1035] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:17.214453 9211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:17.214600 9211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:17.216743 9211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Sending DeleteTablet for 3 replicas of tablet 44bfba5c4c024c44b6ea9ef34decb991
I20260808 15:52:17.217341 9399 tablet_service.cc:1514] Processing DeleteTablet for tablet 44bfba5c4c024c44b6ea9ef34decb991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:17 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:57378
I20260808 15:52:17.217438 9805 tablet_service.cc:1514] Processing DeleteTablet for tablet 44bfba5c4c024c44b6ea9ef34decb991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:17 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56180
I20260808 15:52:17.217602 9596 tablet_service.cc:1514] Processing DeleteTablet for tablet 44bfba5c4c024c44b6ea9ef34decb991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:17 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59874
I20260808 15:52:17.217797 10069 tablet_replica.cc:330] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: stopping tablet replica
I20260808 15:52:17.217845 10068 tablet_replica.cc:330] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: stopping tablet replica
I20260808 15:52:17.217896 10070 tablet_replica.cc:330] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: stopping tablet replica
I20260808 15:52:17.217979 10068 raft_consensus.cc:2229] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:17.218029 10070 raft_consensus.cc:2229] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:17.217988 9211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:52:17.218114 10068 raft_consensus.cc:2258] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:17.218165 10070 raft_consensus.cc:2258] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:17.218297 10069 raft_consensus.cc:2229] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 LEADER]: Raft consensus shutting down.
W20260808 15:52:17.218406 9211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:17.218611 10069 raft_consensus.cc:2258] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:17.218873 10070 ts_tablet_manager.cc:1905] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:17.218880 10068 ts_tablet_manager.cc:1905] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:17.219367 10069 ts_tablet_manager.cc:1905] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:17.221431 10070 ts_tablet_manager.cc:1918] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:17.221520 10070 log.cc:1196] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/44bfba5c4c024c44b6ea9ef34decb991
I20260808 15:52:17.221767 10068 ts_tablet_manager.cc:1918] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:17.221832 10070 ts_tablet_manager.cc:1939] T 44bfba5c4c024c44b6ea9ef34decb991 P 257d55a34238437184d068e0e146bd56: Deleting consensus metadata
I20260808 15:52:17.221856 10068 log.cc:1196] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/44bfba5c4c024c44b6ea9ef34decb991
I20260808 15:52:17.222174 10068 ts_tablet_manager.cc:1939] T 44bfba5c4c024c44b6ea9ef34decb991 P e8a9269d2c6044668385fd7f0081b3b7: Deleting consensus metadata
I20260808 15:52:17.222735 9190 catalog_manager.cc:4841] TS 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191): tablet 44bfba5c4c024c44b6ea9ef34decb991 (table test-table [id=f61d81965dcb4999bd55e15b87bd325e]) successfully deleted
I20260808 15:52:17.223044 9190 catalog_manager.cc:4841] TS e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179): tablet 44bfba5c4c024c44b6ea9ef34decb991 (table test-table [id=f61d81965dcb4999bd55e15b87bd325e]) successfully deleted
I20260808 15:52:17.223606 9399 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1852ebac04accb84718b902f0a39e (DEFAULT_TABLE table=test-table [id=ae404af159e149378378c6cea85f2d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:17.223671 9596 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1852ebac04accb84718b902f0a39e (DEFAULT_TABLE table=test-table [id=ae404af159e149378378c6cea85f2d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:17.223752 9805 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1852ebac04accb84718b902f0a39e (DEFAULT_TABLE table=test-table [id=ae404af159e149378378c6cea85f2d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:17.223857 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1852ebac04accb84718b902f0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:17.223956 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1852ebac04accb84718b902f0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:17.224000 9805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1852ebac04accb84718b902f0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:17.225179 10069 ts_tablet_manager.cc:1918] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:17.225286 10069 log.cc:1196] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/44bfba5c4c024c44b6ea9ef34decb991
I20260808 15:52:17.225672 10069 ts_tablet_manager.cc:1939] T 44bfba5c4c024c44b6ea9ef34decb991 P 32ae52a345af4ac0a5d71a528ef09401: Deleting consensus metadata
I20260808 15:52:17.226462 9937 tablet_bootstrap.cc:492] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7: Bootstrap starting.
I20260808 15:52:17.226769 9935 tablet_bootstrap.cc:492] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Bootstrap starting.
I20260808 15:52:17.226928 9192 catalog_manager.cc:4841] TS 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551): tablet 44bfba5c4c024c44b6ea9ef34decb991 (table test-table [id=f61d81965dcb4999bd55e15b87bd325e]) successfully deleted
I20260808 15:52:17.227440 9937 tablet_bootstrap.cc:654] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:17.227578 9935 tablet_bootstrap.cc:654] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:17.228447 9939 tablet_bootstrap.cc:492] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Bootstrap starting.
I20260808 15:52:17.228730 9935 tablet_bootstrap.cc:492] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: No bootstrap required, opened a new log
I20260808 15:52:17.228824 9935 ts_tablet_manager.cc:1397] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 15:52:17.228991 9937 tablet_bootstrap.cc:492] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7: No bootstrap required, opened a new log
I20260808 15:52:17.229074 9937 ts_tablet_manager.cc:1397] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:17.229277 9935 raft_consensus.cc:348] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.229390 9935 raft_consensus.cc:374] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:17.229426 9935 raft_consensus.cc:729] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae52a345af4ac0a5d71a528ef09401, State: Initialized, Role: FOLLOWER
I20260808 15:52:17.229525 9937 raft_consensus.cc:348] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.229547 9935 consensus_queue.cc:260] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.229693 9939 tablet_bootstrap.cc:654] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:17.229642 9937 raft_consensus.cc:374] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:17.229776 9937 raft_consensus.cc:729] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a9269d2c6044668385fd7f0081b3b7, State: Initialized, Role: FOLLOWER
I20260808 15:52:17.229832 9935 ts_tablet_manager.cc:1428] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 15:52:17.229904 9937 consensus_queue.cc:260] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.230248 9937 ts_tablet_manager.cc:1428] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 15:52:17.231596 9939 tablet_bootstrap.cc:492] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: No bootstrap required, opened a new log
I20260808 15:52:17.231686 9939 ts_tablet_manager.cc:1397] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:17.232178 9939 raft_consensus.cc:348] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.232263 9939 raft_consensus.cc:374] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:17.232285 9939 raft_consensus.cc:729] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257d55a34238437184d068e0e146bd56, State: Initialized, Role: FOLLOWER
I20260808 15:52:17.232401 9939 consensus_queue.cc:260] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.232679 9939 ts_tablet_manager.cc:1428] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 15:52:17.375689 9949 raft_consensus.cc:482] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:17.375875 9949 raft_consensus.cc:504] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.377231 9949 leader_election.cc:288] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191), 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551)
I20260808 15:52:17.381350 10077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.377406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:39191 (local address 127.30.76.65:49603)
0808 15:52:17.377750 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:17.377767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:17.377875 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:17.378181 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:17.378184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:17.378204 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:17.378335 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.378342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.379175 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.379178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.380155 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.380165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.380855 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.380859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.380996 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.381022 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 15:52:17.381112 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 15:52:17.381191 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":69,"thread_start_us":95,"threads_started":1}
I20260808 15:52:17.381337 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.377594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:35053 (local address 127.30.76.67:37551)
0808 15:52:17.377785 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:17.377793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:17.378050 (+ 257us) server_negotiation.cc:407] Connection header received
0808 15:52:17.378191 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:17.378194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:17.378239 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:17.378320 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:17.378592 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.379077 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.380325 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.380734 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.380873 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.380943 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:17.381132 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 15:52:17.381203 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 15:52:17.381369 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.377498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:37551 (local address 127.30.76.65:35053)
0808 15:52:17.377943 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:17.377958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:17.378060 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:17.378345 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:17.378347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:17.378363 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:17.378472 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.378479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.379216 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.379219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.380165 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.380176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.380881 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.380884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.381001 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.381020 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 15:52:17.381115 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 15:52:17.381188 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":69,"thread_start_us":133,"threads_started":1}
I20260808 15:52:17.381354 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.377538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:49603 (local address 127.30.76.66:39191)
0808 15:52:17.377812 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:17.377817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:17.377836 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:52:17.377999 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:17.378005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:17.378064 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:17.378161 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:17.378455 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.379044 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.380284 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.380706 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.380825 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.380885 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:17.381140 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 15:52:17.381209 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260808 15:52:17.381933 9834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1852ebac04accb84718b902f0a39e" candidate_uuid: "e8a9269d2c6044668385fd7f0081b3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ae52a345af4ac0a5d71a528ef09401" is_pre_election: true
I20260808 15:52:17.382189 9834 raft_consensus.cc:2454] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a9269d2c6044668385fd7f0081b3b7 in term 0.
I20260808 15:52:17.382202 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1852ebac04accb84718b902f0a39e" candidate_uuid: "e8a9269d2c6044668385fd7f0081b3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56" is_pre_election: true
I20260808 15:52:17.382325 9616 raft_consensus.cc:2454] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a9269d2c6044668385fd7f0081b3b7 in term 0.
I20260808 15:52:17.382716 9328 leader_election.cc:302] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d55a34238437184d068e0e146bd56, e8a9269d2c6044668385fd7f0081b3b7; no voters:
I20260808 15:52:17.382988 9949 raft_consensus.cc:2781] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:17.383065 9949 raft_consensus.cc:482] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:17.383121 9949 raft_consensus.cc:3037] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:17.384155 9949 raft_consensus.cc:504] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.384500 9949 leader_election.cc:288] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [CANDIDATE]: Term 1 election: Requested vote from peers 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191), 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551)
I20260808 15:52:17.384835 9616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1852ebac04accb84718b902f0a39e" candidate_uuid: "e8a9269d2c6044668385fd7f0081b3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d55a34238437184d068e0e146bd56"
I20260808 15:52:17.384963 9616 raft_consensus.cc:3037] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:17.385850 9616 raft_consensus.cc:2454] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a9269d2c6044668385fd7f0081b3b7 in term 1.
I20260808 15:52:17.386122 9834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1852ebac04accb84718b902f0a39e" candidate_uuid: "e8a9269d2c6044668385fd7f0081b3b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ae52a345af4ac0a5d71a528ef09401"
I20260808 15:52:17.386168 9328 leader_election.cc:302] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d55a34238437184d068e0e146bd56, e8a9269d2c6044668385fd7f0081b3b7; no voters:
I20260808 15:52:17.386281 9834 raft_consensus.cc:3037] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:17.386381 9949 raft_consensus.cc:2781] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:17.386644 9949 raft_consensus.cc:686] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 1 LEADER]: Becoming Leader. State: Replica: e8a9269d2c6044668385fd7f0081b3b7, State: Running, Role: LEADER
I20260808 15:52:17.386896 9949 consensus_queue.cc:237] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } }
I20260808 15:52:17.387446 9834 raft_consensus.cc:2454] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a9269d2c6044668385fd7f0081b3b7 in term 1.
I20260808 15:52:17.389307 9209 catalog_manager.cc:5495] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65). New cstate: current_term: 1 leader_uuid: "e8a9269d2c6044668385fd7f0081b3b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a9269d2c6044668385fd7f0081b3b7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 } health_report { overall_health: UNKNOWN } } }
I20260808 15:52:17.427404 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.423456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57390 (local address 127.30.76.65:42179)
0808 15:52:17.423827 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:17.423834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:17.423845 (+ 11us) server_negotiation.cc:407] Connection header received
0808 15:52:17.423913 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:17.423915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:17.423967 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:17.424096 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:17.424339 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.424865 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.425751 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.426049 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.426177 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.426445 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:17.426992 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:17.427151 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:17.427188 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 15:52:17.427239 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":190,"threads_started":1}
I20260808 15:52:17.437373 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.433837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:37551 (local address 127.30.76.65:59817)
0808 15:52:17.434204 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:17.434216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:17.434297 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:17.434512 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:17.434516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:17.434524 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:52:17.434667 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.434673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.435455 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.435461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.436164 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.436173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.436629 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.436632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.436727 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.436774 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:52:17.437085 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:52:17.437095 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 15:52:17.437161 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 15:52:17.437237 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":99,"threads_started":1}
I20260808 15:52:17.437348 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.433940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:59817 (local address 127.30.76.67:37551)
0808 15:52:17.434088 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:17.434092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:17.434283 (+ 191us) server_negotiation.cc:407] Connection header received
0808 15:52:17.434362 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:17.434364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:17.434406 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:17.434497 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:17.434787 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.435343 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.436292 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.436508 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.436626 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.436919 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:17.436989 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:17.437079 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:17.437169 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 15:52:17.437215 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:52:17.443168 9616 raft_consensus.cc:1261] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Refusing update from remote peer e8a9269d2c6044668385fd7f0081b3b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:17.443441 9834 raft_consensus.cc:1261] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Refusing update from remote peer e8a9269d2c6044668385fd7f0081b3b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:17.443648 9949 consensus_queue.cc:1035] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257d55a34238437184d068e0e146bd56" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:17.443956 10081 consensus_queue.cc:1035] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ae52a345af4ac0a5d71a528ef09401" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 37551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:17.461742 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.458044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:33909 (local address 127.30.76.65:42179)
0808 15:52:17.458169 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:17.458174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:17.458405 (+ 231us) server_negotiation.cc:407] Connection header received
0808 15:52:17.458525 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:17.458528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:17.458570 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:17.458665 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:17.458918 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.459423 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.460487 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.460844 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.460961 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.461232 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:17.461310 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:17.461396 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:17.461562 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 15:52:17.461619 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 15:52:17.461755 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.457960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42179 (local address 127.30.76.67:33909)
0808 15:52:17.458321 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:17.458337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:17.458443 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:17.458676 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:17.458679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:17.458687 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:52:17.458787 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.458792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.459562 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.459566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.460361 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:17.460372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:17.460963 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:17.460966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:17.461058 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:17.461092 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:52:17.461402 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:52:17.461443 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 15:52:17.461533 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 15:52:17.461634 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":95,"threads_started":1}
I20260808 15:52:17.483245 9211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:17.483386 9211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:17.485455 9211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2301fa59c9a4432c9e12e3ba5a21bf7f: Sending DeleteTablet for 3 replicas of tablet 44c1852ebac04accb84718b902f0a39e
I20260808 15:52:17.486073 9399 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1852ebac04accb84718b902f0a39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:17 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:57378
I20260808 15:52:17.486068 9596 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1852ebac04accb84718b902f0a39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:17 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59874
I20260808 15:52:17.486254 10070 tablet_replica.cc:330] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: stopping tablet replica
I20260808 15:52:17.486254 10068 tablet_replica.cc:330] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7: stopping tablet replica
I20260808 15:52:17.486363 10070 raft_consensus.cc:2229] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:17.486363 10068 raft_consensus.cc:2229] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:17.486469 10070 raft_consensus.cc:2258] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:17.486672 10068 raft_consensus.cc:2258] T 44c1852ebac04accb84718b902f0a39e P e8a9269d2c6044668385fd7f0081b3b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:17.486701 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 9310
I20260808 15:52:17.487355 9805 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1852ebac04accb84718b902f0a39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:17 UTC) from {username='oryx', principal='oryx/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56180
I20260808 15:52:17.487612 10069 tablet_replica.cc:330] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: stopping tablet replica
I20260808 15:52:17.487753 10069 raft_consensus.cc:2229] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:17.487855 10069 raft_consensus.cc:2258] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:17.490675 10069 ts_tablet_manager.cc:1905] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:17.494011 10069 ts_tablet_manager.cc:1918] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:52:17.494109 10069 log.cc:1196] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/44c1852ebac04accb84718b902f0a39e
I20260808 15:52:17.494504 10069 ts_tablet_manager.cc:1939] T 44c1852ebac04accb84718b902f0a39e P 32ae52a345af4ac0a5d71a528ef09401: Deleting consensus metadata
I20260808 15:52:17.495270 9192 catalog_manager.cc:4841] TS 32ae52a345af4ac0a5d71a528ef09401 (127.30.76.67:37551): tablet 44c1852ebac04accb84718b902f0a39e (table test-table [id=ae404af159e149378378c6cea85f2d5e]) successfully deleted
I20260808 15:52:17.496485 10070 ts_tablet_manager.cc:1905] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:17.498368 10070 ts_tablet_manager.cc:1918] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:52:17.498437 10070 log.cc:1196] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/44c1852ebac04accb84718b902f0a39e
I20260808 15:52:17.498698 10070 ts_tablet_manager.cc:1939] T 44c1852ebac04accb84718b902f0a39e P 257d55a34238437184d068e0e146bd56: Deleting consensus metadata
I20260808 15:52:17.499388 9190 catalog_manager.cc:4841] TS 257d55a34238437184d068e0e146bd56 (127.30.76.66:39191): tablet 44c1852ebac04accb84718b902f0a39e (table test-table [id=ae404af159e149378378c6cea85f2d5e]) successfully deleted
W20260808 15:52:17.503219 9190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:42179 (error 108)
I20260808 15:52:17.503530 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 9499
I20260808 15:52:17.506953 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:17.505942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42179 (local address 127.0.0.1:57400)
0808 15:52:17.506728 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:17.506818 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":636,"threads_started":1}
W20260808 15:52:17.507793 9190 catalog_manager.cc:4584] TS e8a9269d2c6044668385fd7f0081b3b7 (127.30.76.65:42179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44c1852ebac04accb84718b902f0a39e: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42179: connect: Connection refused (error 111)
I20260808 15:52:17.513052 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 9711
I20260808 15:52:17.524230 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 9164
2026-08-08T15:52:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4001 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:17 dist-test-slave-sb2z krb5kdc[10111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:17 dist-test-slave-sb2z krb5kdc[10111](info): set up 2 sockets
Aug 08 15:52:17 dist-test-slave-sb2z krb5kdc[10111](info): commencing operation
krb5kdc: starting...
W20260808 15:52:18.740757 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:18 dist-test-slave-sb2z krb5kdc[10111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:52:18.864761 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:33243
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:33243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:18.978348 10133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:18.978579 10133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:18.978617 10133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:18.982656 10133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:18.982710 10133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:18.982729 10133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:18.982746 10133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:18.982760 10133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:18.987883 10133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:33243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:33243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:18.989032 10133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:18.990150 10133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:18.998612 10140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:18.998593 10145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:18.998594 10142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:18.998914 10133 server_base.cc:1034] running on GCE node
I20260808 15:52:18.999598 10133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:19.000788 10133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:19.001995 10133 hybrid_clock.cc:648] HybridClock initialized: now 1786204339001964 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T15:52:19Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1490 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:19 dist-test-slave-sb2z krb5kdc[10154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:19 dist-test-slave-sb2z krb5kdc[10154](info): set up 2 sockets
Aug 08 15:52:19 dist-test-slave-sb2z krb5kdc[10154](info): commencing operation
krb5kdc: starting...
W20260808 15:52:20.239074 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:20 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:52:20.364569 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36367
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:36367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:20.474113 10168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:20.474407 10168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:20.474483 10168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:20.478426 10168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:20.478519 10168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:20.478562 10168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:20.478621 10168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:20.478645 10168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:20.483801 10168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:36367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:20.484802 10168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:20.485805 10168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:20.494009 10177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:20.494014 10176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:20.494014 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:20.494323 10168 server_base.cc:1034] running on GCE node
I20260808 15:52:20.494918 10168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:20.496024 10168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:20.497217 10168 hybrid_clock.cc:648] HybridClock initialized: now 1786204340497195 us; error 45 us; skew 500 ppm
Aug 08 15:52:20 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:20.500018 10168 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:52:20.500835 10168 webserver.cc:460] Webserver started at http://127.30.76.126:45203/ using document root <none> and password file <none>
I20260808 15:52:20.501397 10168 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:20.501459 10168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:20.501693 10168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:20.503734 10168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "dec3cfc8ab244949a69a49c3a373b9e0"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "9fc425a3433cc15b05b41ea10f4324e3"
server_key_iv: "54fdf4561803fde177a2a59b611820f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.504302 10168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "dec3cfc8ab244949a69a49c3a373b9e0"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "9fc425a3433cc15b05b41ea10f4324e3"
server_key_iv: "54fdf4561803fde177a2a59b611820f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.507827 10168 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:20.510090 10187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:20.511237 10168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:20.511382 10168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "dec3cfc8ab244949a69a49c3a373b9e0"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "9fc425a3433cc15b05b41ea10f4324e3"
server_key_iv: "54fdf4561803fde177a2a59b611820f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.511497 10168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:20.532347 10168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:20.536347 10168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:20.536571 10168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:20.544035 10168 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:36367
I20260808 15:52:20.544093 10260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:36367 every 8 connection(s)
I20260808 15:52:20.545080 10168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:20.548022 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:20.550660 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 10168
I20260808 15:52:20.550797 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:20.551121 31025 external_mini_cluster.cc:1428] Setting key b5ee0f896916eb712f9e348b25690ec9
I20260808 15:52:20.554296 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Bootstrap starting.
I20260808 15:52:20.556591 10261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:20.557349 10261 log.cc:824] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:20.559219 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: No bootstrap required, opened a new log
Aug 08 15:52:20 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:20.562649 10261 raft_consensus.cc:348] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:20.562866 10261 raft_consensus.cc:374] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:20.562927 10261 raft_consensus.cc:729] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec3cfc8ab244949a69a49c3a373b9e0, State: Initialized, Role: FOLLOWER
I20260808 15:52:20.563369 10261 consensus_queue.cc:260] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:20.563614 10261 raft_consensus.cc:388] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:20.563705 10261 raft_consensus.cc:482] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:20.563813 10261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:20.564922 10261 raft_consensus.cc:504] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:20.565304 10261 leader_election.cc:302] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dec3cfc8ab244949a69a49c3a373b9e0; no voters:
I20260808 15:52:20.565670 10261 leader_election.cc:288] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:20.565793 10267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:20.566022 10267 raft_consensus.cc:686] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 LEADER]: Becoming Leader. State: Replica: dec3cfc8ab244949a69a49c3a373b9e0, State: Running, Role: LEADER
I20260808 15:52:20.566323 10267 consensus_queue.cc:237] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:20.566941 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:20.552676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.30.76.126:36367)
0808 15:52:20.553117 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:20.553132 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:20.553167 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:52:20.553777 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:20.553794 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:20.554113 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:20.554447 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:20.555294 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:20.556350 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:20.557307 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:20.557829 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:20.558002 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:20.560858 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:20.560863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:20.560876 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:20.560904 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:20.563323 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:20.563868 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:20.563874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:20.563896 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:20.564035 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:20.564342 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:20.564344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:20.564345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:20.564733 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:20.564883 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:20.565622 (+ 739us) server_negotiation.cc:299] Negotiation successful
0808 15:52:20.565940 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
I20260808 15:52:20.566948 10261 sys_catalog.cc:564] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:20.568172 10270 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dec3cfc8ab244949a69a49c3a373b9e0. Latest consensus state: current_term: 1 leader_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } } }
I20260808 15:52:20.568305 10270 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:20.568176 10268 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } } }
I20260808 15:52:20.568595 10268 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:20.569481 10274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:20.572125 10274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:20.577557 10274 catalog_manager.cc:1347] Generated new cluster ID: 85525ce820c84c51986aaa8f0594d145
I20260808 15:52:20.577644 10274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:20.592309 10274 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:20.593526 10274 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:20.601364 10274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Generated new TSK 0
I20260808 15:52:20.602128 10274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:52:20.638334 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36367
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:20.753659 10292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:20.753974 10292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:20.754035 10292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:20.758114 10292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:20.758174 10292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:20.758230 10292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:20.763293 10292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:20.764432 10292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:20.765413 10292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:20.775175 10308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:20.775178 10303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:20.775154 10302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:20.775383 10292 server_base.cc:1034] running on GCE node
I20260808 15:52:20.776111 10292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:20.776765 10292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:20.777956 10292 hybrid_clock.cc:648] HybridClock initialized: now 1786204340777939 us; error 29 us; skew 500 ppm
Aug 08 15:52:20 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:20.780838 10292 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:52:20.781772 10292 webserver.cc:460] Webserver started at http://127.30.76.65:35463/ using document root <none> and password file <none>
I20260808 15:52:20.782346 10292 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:20.782442 10292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:20.782680 10292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:20.784812 10292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "5d5f07bd5e1d49efaa04de4ffa705406"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "a16a9d3e39e7e5dfbac4d00173c60282"
server_key_iv: "a678507e0c383d1caf4ea3eebf3ce924"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.785403 10292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "5d5f07bd5e1d49efaa04de4ffa705406"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "a16a9d3e39e7e5dfbac4d00173c60282"
server_key_iv: "a678507e0c383d1caf4ea3eebf3ce924"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.789346 10292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:52:20.791757 10315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:20.793026 10292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:20.793195 10292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "5d5f07bd5e1d49efaa04de4ffa705406"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "a16a9d3e39e7e5dfbac4d00173c60282"
server_key_iv: "a678507e0c383d1caf4ea3eebf3ce924"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.793311 10292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:20.806046 10292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:20.809258 10292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:20.809444 10292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:20.810053 10292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:20.810875 10292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:20.810923 10292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:20.810961 10292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:20.810983 10292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:20.819770 10292 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:40279
I20260808 15:52:20.819854 10466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:40279 every 8 connection(s)
I20260808 15:52:20.820647 10292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:20.825073 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 10292
I20260808 15:52:20.825208 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:20.825521 31025 external_mini_cluster.cc:1428] Setting key 8b40b71413cdcff590eefa2b59ec28a8
Aug 08 15:52:20 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:20.834937 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:20.822374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:50445 (local address 127.30.76.126:36367)
0808 15:52:20.822514 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:20.822518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:20.823285 (+ 767us) server_negotiation.cc:407] Connection header received
0808 15:52:20.824553 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:20.824557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:20.824617 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:20.824730 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:20.826124 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:20.826761 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:20.828034 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:20.828275 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:20.828393 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:20.831108 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:20.831112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:20.831115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:20.831141 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:20.832773 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:20.833301 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:20.833305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:20.833307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:20.833398 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:20.833744 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:20.833748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:20.833750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:20.834011 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:20.834131 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:20.834622 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 15:52:20.834763 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:52:20.835811 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:20.822716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:50445)
0808 15:52:20.823135 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:20.823175 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:20.824335 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:20.824929 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:20.824935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:20.825367 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:20.825961 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:20.825980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:20.826900 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:20.826903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:20.827902 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:20.827912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:20.828436 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:20.828441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:20.828609 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:20.829219 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:20.829245 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:20.830936 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:20.832901 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:20.832907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:20.832923 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:20.833191 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:20.833493 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:20.833497 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:20.833499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:20.833645 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:20.834130 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:20.834146 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:20.834386 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 15:52:20.834579 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 15:52:20.834895 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260808 15:52:20.837175 10467 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36367
I20260808 15:52:20.837415 10467 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:20.837973 10467 heartbeater.cc:511] Master 127.30.76.126:36367 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:52:20.839675 10212 ts_manager.cc:194] Registered new tserver with Master: 5d5f07bd5e1d49efaa04de4ffa705406 (127.30.76.65:40279)
I20260808 15:52:20.840792 10212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:50445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:52:20.852470 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36367
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:20.962387 10473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:20.962632 10473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:20.962674 10473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:20.966500 10473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:20.966547 10473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:20.966662 10473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:20.971916 10473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:20.972806 10473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:20.973726 10473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:20.985960 10482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:20.986135 10485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:20.986371 10473 server_base.cc:1034] running on GCE node
W20260808 15:52:20.986423 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:20.986989 10473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:20.987778 10473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:20.989001 10473 hybrid_clock.cc:648] HybridClock initialized: now 1786204340988966 us; error 53 us; skew 500 ppm
Aug 08 15:52:20 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:20.991784 10473 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:52:20.992688 10473 webserver.cc:460] Webserver started at http://127.30.76.66:43653/ using document root <none> and password file <none>
I20260808 15:52:20.993247 10473 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:20.993338 10473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:20.993558 10473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:20.995659 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "3c6387035f874a09a5694dfeac883405"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "90febc67c50be0ab3a812f3907b3774b"
server_key_iv: "b7a996f91769816f62694d1ba69d0dc2"
server_key_version: "encryptionkey@0"
I20260808 15:52:20.996318 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "3c6387035f874a09a5694dfeac883405"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "90febc67c50be0ab3a812f3907b3774b"
server_key_iv: "b7a996f91769816f62694d1ba69d0dc2"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.000144 10473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:52:21.002534 10493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.003839 10473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:21.003965 10473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3c6387035f874a09a5694dfeac883405"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "90febc67c50be0ab3a812f3907b3774b"
server_key_iv: "b7a996f91769816f62694d1ba69d0dc2"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.004069 10473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:21.030778 10473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:21.034096 10473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:21.034354 10473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:21.035028 10473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:21.035934 10473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:21.035993 10473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.036041 10473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:21.036063 10473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.045092 10473 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:44215
I20260808 15:52:21.045116 10645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:44215 every 8 connection(s)
I20260808 15:52:21.045998 10473 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:21.048830 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 10473
I20260808 15:52:21.048976 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:21.049337 31025 external_mini_cluster.cc:1428] Setting key bad4964def21ca8110ab05132d995d61
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204340, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:52:21.060848 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.047908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:36119 (local address 127.30.76.126:36367)
0808 15:52:21.048071 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:21.048075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:21.048992 (+ 917us) server_negotiation.cc:407] Connection header received
0808 15:52:21.050447 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:21.050450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:21.050493 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:21.050563 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:21.051913 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.052735 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.053920 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.054306 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.054432 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.057007 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:21.057010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:21.057012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:21.057035 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:21.058557 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:21.059175 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:21.059178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:21.059179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:21.059256 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:21.059677 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:21.059679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:21.059680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:21.059849 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:21.059928 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:21.060568 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 15:52:21.060715 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:52:21.061829 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.048293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:36119)
0808 15:52:21.048743 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:21.048786 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:21.050226 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:21.050771 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:21.050793 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:21.051197 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:21.051755 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.051771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.052895 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.052898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.053791 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.053801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.054545 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.054548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.054694 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.055268 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:21.055283 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:21.056837 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:21.058672 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:21.058679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:21.058696 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:21.059067 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:21.059351 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:21.059355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:21.059357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:21.059576 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:21.059964 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:21.059974 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:21.060309 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 15:52:21.060501 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 15:52:21.060857 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:52:21.063308 10646 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36367
I20260808 15:52:21.063575 10646 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:21.064199 10646 heartbeater.cc:511] Master 127.30.76.126:36367 requested a full tablet report, sending...
I20260808 15:52:21.065339 10212 ts_manager.cc:194] Registered new tserver with Master: 3c6387035f874a09a5694dfeac883405 (127.30.76.66:44215)
I20260808 15:52:21.065918 10212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:36119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:52:21.080622 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36367
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:21.191324 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:21.191634 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:21.191716 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:21.195595 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:21.195660 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:21.195724 10654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:21.200901 10654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:21.201794 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:21.202698 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:21.212055 10665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.212055 10661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.212116 10663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:21.212313 10654 server_base.cc:1034] running on GCE node
I20260808 15:52:21.212725 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:21.213343 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:21.214536 10654 hybrid_clock.cc:648] HybridClock initialized: now 1786204341214524 us; error 35 us; skew 500 ppm
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204341, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:21.217523 10654 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:52:21.218401 10654 webserver.cc:460] Webserver started at http://127.30.76.67:46519/ using document root <none> and password file <none>
I20260808 15:52:21.218961 10654 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:21.219053 10654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:21.219281 10654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:21.221345 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "eec577c912aa46b8845f7b4687c4c2a5"
format_stamp: "Formatted at 2026-08-08 15:52:21 on dist-test-slave-sb2z"
server_key: "359f0b3f1d1383372b3fe4ed80d9185d"
server_key_iv: "d485cb23fe9432ed29b6bbe3501d7e99"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.221911 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "eec577c912aa46b8845f7b4687c4c2a5"
format_stamp: "Formatted at 2026-08-08 15:52:21 on dist-test-slave-sb2z"
server_key: "359f0b3f1d1383372b3fe4ed80d9185d"
server_key_iv: "d485cb23fe9432ed29b6bbe3501d7e99"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.225827 10654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:52:21.228183 10676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.229353 10654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:21.229486 10654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "eec577c912aa46b8845f7b4687c4c2a5"
format_stamp: "Formatted at 2026-08-08 15:52:21 on dist-test-slave-sb2z"
server_key: "359f0b3f1d1383372b3fe4ed80d9185d"
server_key_iv: "d485cb23fe9432ed29b6bbe3501d7e99"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.229610 10654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:21.264230 10654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:21.267866 10654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:21.268077 10654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:21.268697 10654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:21.269639 10654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:21.269688 10654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.269755 10654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:21.269790 10654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.278234 10654 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44461
I20260808 15:52:21.278306 10829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44461 every 8 connection(s)
I20260808 15:52:21.279137 10654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:21.287856 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 10654
I20260808 15:52:21.287985 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:21.288290 31025 external_mini_cluster.cc:1428] Setting key 1fb521153739a91d0115cec7aaf33277
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204341, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:21.292618 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.280936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:35373 (local address 127.30.76.126:36367)
0808 15:52:21.281072 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:21.281076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:21.281903 (+ 827us) server_negotiation.cc:407] Connection header received
0808 15:52:21.283124 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:21.283127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:21.283189 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:21.283277 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:21.284462 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.285005 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.286074 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.286291 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.286374 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.289002 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:21.289005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:21.289007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:21.289031 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:21.290727 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:21.291205 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:21.291208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:21.291210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:21.291327 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:21.291722 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:21.291725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:21.291726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:21.291893 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:21.291971 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:21.292412 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 15:52:21.292504 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:52:21.293438 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.281299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:35373)
0808 15:52:21.281719 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:21.281759 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:21.282908 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:21.283426 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:21.283432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:21.283833 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:21.284300 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.284314 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.285119 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.285122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.285960 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.285968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.286422 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.286425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.286558 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.287160 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:21.287176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:21.288812 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:21.290835 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:21.290840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:21.290851 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:21.291091 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:21.291431 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:21.291434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:21.291436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:21.291638 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:21.291982 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:21.291987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:21.292206 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 15:52:21.292348 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 15:52:21.292576 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":102,"threads_started":1}
I20260808 15:52:21.294468 10831 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36367
I20260808 15:52:21.294703 10831 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:21.295184 10831 heartbeater.cc:511] Master 127.30.76.126:36367 requested a full tablet report, sending...
I20260808 15:52:21.296177 10212 ts_manager.cc:194] Registered new tserver with Master: eec577c912aa46b8845f7b4687c4c2a5 (127.30.76.67:44461)
I20260808 15:52:21.296857 10212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:35373
I20260808 15:52:21.302426 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab.
Entry for principal oryx/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab.
WARNING: no policy specified for oryx/127.30.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab.
Entry for principal oryx/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab.
WARNING: no policy specified for oryx/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab.
Entry for principal oryx/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab.
I20260808 15:52:21.378173 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 10292
I20260808 15:52:21.394202 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 10473
I20260808 15:52:21.406129 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 10654
I20260808 15:52:21.417032 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 10168
I20260808 15:52:21.429999 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36367
--webserver_interface=127.30.76.126
--webserver_port=45203
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:36367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:21.542567 10841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:21.542848 10841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:21.542928 10841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:21.546881 10841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:21.546968 10841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:21.547006 10841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:21.547063 10841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:21.547104 10841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:21.552255 10841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:36367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=45203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:21.553273 10841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:21.554302 10841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:21.562880 10849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.562966 10851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.562880 10848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:21.562934 10841 server_base.cc:1034] running on GCE node
I20260808 15:52:21.563795 10841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:21.564800 10841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:21.565943 10841 hybrid_clock.cc:648] HybridClock initialized: now 1786204341565929 us; error 26 us; skew 500 ppm
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204341, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:21.568678 10841 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:52:21.569454 10841 webserver.cc:460] Webserver started at http://127.30.76.126:45203/ using document root <none> and password file <none>
I20260808 15:52:21.569952 10841 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:21.570024 10841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:21.573683 10841 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:21.575695 10859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.577210 10841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:21.577361 10841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "dec3cfc8ab244949a69a49c3a373b9e0"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "9fc425a3433cc15b05b41ea10f4324e3"
server_key_iv: "54fdf4561803fde177a2a59b611820f3"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.577870 10841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:21.592602 10841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:21.595155 10841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:21.595300 10841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:21.602142 10841 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:36367
I20260808 15:52:21.602224 10933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:36367 every 8 connection(s)
I20260808 15:52:21.603220 10841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:21.606599 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 10841
I20260808 15:52:21.607040 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:40279
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:36367
--webserver_port=35463
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:52:21.607591 10934 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:52:21.609427 10934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Bootstrap starting.
I20260808 15:52:21.633412 10934 log.cc:824] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:21.641675 10934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:52:21.642096 10934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Bootstrap complete.
I20260808 15:52:21.647893 10934 raft_consensus.cc:348] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:21.648394 10934 raft_consensus.cc:729] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec3cfc8ab244949a69a49c3a373b9e0, State: Initialized, Role: FOLLOWER
I20260808 15:52:21.648922 10934 consensus_queue.cc:260] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:21.649065 10934 raft_consensus.cc:388] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:21.649142 10934 raft_consensus.cc:482] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:21.649240 10934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:52:21.651721 10934 raft_consensus.cc:504] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:21.652112 10934 leader_election.cc:302] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dec3cfc8ab244949a69a49c3a373b9e0; no voters:
I20260808 15:52:21.652467 10934 leader_election.cc:288] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:52:21.652595 10940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:52:21.652890 10940 raft_consensus.cc:686] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [term 2 LEADER]: Becoming Leader. State: Replica: dec3cfc8ab244949a69a49c3a373b9e0, State: Running, Role: LEADER
I20260808 15:52:21.653194 10940 consensus_queue.cc:237] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } }
I20260808 15:52:21.653682 10934 sys_catalog.cc:564] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:21.654167 10945 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dec3cfc8ab244949a69a49c3a373b9e0. Latest consensus state: current_term: 2 leader_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } } }
I20260808 15:52:21.654304 10945 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:21.654559 10944 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3cfc8ab244949a69a49c3a373b9e0" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36367 } } }
I20260808 15:52:21.654649 10944 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:21.654721 10948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:21.657760 10948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:21.658531 10948 catalog_manager.cc:1259] Loaded cluster ID: 85525ce820c84c51986aaa8f0594d145
I20260808 15:52:21.658595 10948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:21.660890 10948 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:21.661572 10948 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dec3cfc8ab244949a69a49c3a373b9e0: Loaded TSK: 0
I20260808 15:52:21.662730 10948 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:52:21.736647 10938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:21.736948 10938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:21.737054 10938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:21.741267 10938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:21.741355 10938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:21.741449 10938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:21.746871 10938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.65.keytab
--principal=oryx/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:40279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=35463
--tserver_master_addrs=127.30.76.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:21.748142 10938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:21.749265 10938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:21.760388 10970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.760551 10969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.760388 10972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:21.760957 10938 server_base.cc:1034] running on GCE node
I20260808 15:52:21.761435 10938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:21.762270 10938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:21.763490 10938 hybrid_clock.cc:648] HybridClock initialized: now 1786204341763470 us; error 27 us; skew 500 ppm
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204341, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:21.767015 10938 init.cc:377] Logged in from keytab as oryx/127.30.76.65@KRBTEST.COM (short username oryx)
I20260808 15:52:21.768049 10938 webserver.cc:460] Webserver started at http://127.30.76.65:35463/ using document root <none> and password file <none>
I20260808 15:52:21.768632 10938 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:21.768709 10938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:21.772699 10938 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:21.774852 10984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.776237 10938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:21.776386 10938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "5d5f07bd5e1d49efaa04de4ffa705406"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "a16a9d3e39e7e5dfbac4d00173c60282"
server_key_iv: "a678507e0c383d1caf4ea3eebf3ce924"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.776921 10938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:21.798031 10938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:21.801363 10938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:21.801576 10938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:21.802191 10938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:21.803040 10938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:21.803110 10938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.803186 10938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:21.803231 10938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.812273 10938 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:40279
I20260808 15:52:21.812311 11140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:40279 every 8 connection(s)
I20260808 15:52:21.813265 10938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:21.815399 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 10938
I20260808 15:52:21.815930 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:44215
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:36367
--webserver_port=43653
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:52:21.826936 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:21.815342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:38499 (local address 127.30.76.126:36367)
0808 15:52:21.817476 (+ 2134us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:21.817489 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:21.817520 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:52:21.817715 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:21.817724 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:21.818011 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:21.818329 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:21.819515 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.820638 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.821780 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.822245 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.822427 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.826244 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:38499: BlockingRecv error: recv got EOF from 127.30.76.65:38499 (error 108)
Metrics: {"server-negotiator.queue_time_us":1997,"thread_start_us":142,"threads_started":1}
I20260808 15:52:21.827134 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.815314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:38499)
0808 15:52:21.815821 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:21.815873 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:21.817073 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:21.818407 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:21.818415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:21.818867 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:21.819318 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.819331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.820777 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.820780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.821648 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.821658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.822413 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.822417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.822604 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.823248 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:21.823265 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:21.826063 (+ 2798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":173,"threads_started":1}
W20260808 15:52:21.827718 11141 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:36367 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.76.126:36367: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:21.833495 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.828159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:53147)
0808 15:52:21.828356 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:21.828377 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:21.828440 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:21.828680 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:21.828682 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:21.828810 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:21.828895 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.828900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.829901 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.829903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.830437 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.830444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.831008 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.831011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.831077 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.831607 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:21.831618 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:21.833372 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260808 15:52:21.833645 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:21.828285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:53147 (local address 127.30.76.126:36367)
0808 15:52:21.828413 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:21.828418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:21.828435 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:52:21.828524 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:21.828527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:21.828576 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:21.828656 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:21.829005 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.829800 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.830545 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.830893 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.830993 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.833499 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:53147: BlockingRecv error: recv got EOF from 127.30.76.65:53147 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:21.839339 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:21.834145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:57485)
0808 15:52:21.834285 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:21.834302 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:21.834389 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:21.834639 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:21.834645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:21.834822 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:21.834910 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.834916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.835606 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.835610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.836455 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:21.836465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.836898 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.836901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:21.836993 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.837428 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:21.837442 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:21.839207 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:52:21.839427 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:21.834257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:57485 (local address 127.30.76.126:36367)
0808 15:52:21.834359 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:21.834362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:21.834374 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:52:21.834484 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:21.834486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:21.834523 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:21.834586 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:21.835018 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.835470 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.836590 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:21.836794 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:21.836872 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:21.839323 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:57485: BlockingRecv error: recv got EOF from 127.30.76.65:57485 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 15:52:21.839684 11141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:52:21.933027 11147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:21.933331 11147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:21.933444 11147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:21.937569 11147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:21.937644 11147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:21.937726 11147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:21.942871 11147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.66.keytab
--principal=oryx/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:44215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=43653
--tserver_master_addrs=127.30.76.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:21.943881 11147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:21.944896 11147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:21.954555 11153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.954574 11154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:21.954576 11156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:21.954936 11147 server_base.cc:1034] running on GCE node
I20260808 15:52:21.955343 11147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:21.956072 11147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:21.957238 11147 hybrid_clock.cc:648] HybridClock initialized: now 1786204341957222 us; error 25 us; skew 500 ppm
Aug 08 15:52:21 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204341, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:21.960094 11147 init.cc:377] Logged in from keytab as oryx/127.30.76.66@KRBTEST.COM (short username oryx)
I20260808 15:52:21.960938 11147 webserver.cc:460] Webserver started at http://127.30.76.66:43653/ using document root <none> and password file <none>
I20260808 15:52:21.961490 11147 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:21.961585 11147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:21.965371 11147 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 15:52:21.967316 11164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:21.968552 11147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:52:21.968685 11147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3c6387035f874a09a5694dfeac883405"
format_stamp: "Formatted at 2026-08-08 15:52:20 on dist-test-slave-sb2z"
server_key: "90febc67c50be0ab3a812f3907b3774b"
server_key_iv: "b7a996f91769816f62694d1ba69d0dc2"
server_key_version: "encryptionkey@0"
I20260808 15:52:21.969208 11147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:21.997275 11147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:22.000636 11147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:22.000847 11147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:22.001485 11147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:22.002339 11147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:22.002413 11147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:22.002496 11147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:22.002537 11147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:22.012028 11147 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:44215
I20260808 15:52:22.012045 11309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:44215 every 8 connection(s)
I20260808 15:52:22.013120 11147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:22.013695 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 11147
I20260808 15:52:22.014179 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:44461
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:36367
--webserver_port=46519
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:52:22.027326 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.015249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:53171 (local address 127.30.76.126:36367)
0808 15:52:22.015400 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.015404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.016575 (+ 1171us) server_negotiation.cc:407] Connection header received
0808 15:52:22.018519 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.018521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.018579 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.018643 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.020449 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.020979 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.022873 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.023081 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.023161 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.027197 (+ 4036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:53171: BlockingRecv error: recv got EOF from 127.30.76.66:53171 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:52:22.027980 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.015772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:53171)
0808 15:52:22.016375 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.016437 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.018267 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.018843 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.018854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.019471 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.020278 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.020304 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.021121 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.021128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.022728 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.022744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.023248 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.023255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.023501 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.024203 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.024231 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.027071 (+ 2840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":147,"threads_started":1}
W20260808 15:52:22.028488 11310 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:36367 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.76.126:36367: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:22.034039 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.028972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:38131)
0808 15:52:22.029107 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.029123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.029222 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.029475 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.029478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.029635 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.029722 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.029727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.030486 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.030490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.031017 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.031025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.031460 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.031464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.031580 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.032063 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.032074 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.033905 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 15:52:22.034125 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.029053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:38131 (local address 127.30.76.126:36367)
0808 15:52:22.029164 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.029171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.029186 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:22.029326 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.029335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.029375 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.029459 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.029832 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.030378 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.031136 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.031350 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.031460 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.034013 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:38131: BlockingRecv error: recv got EOF from 127.30.76.66:38131 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:22.039866 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.034568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:50775)
0808 15:52:22.034697 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.034711 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.034795 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.035011 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.035014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.035172 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.035253 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.035268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.035999 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.036002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.036844 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.036854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.037299 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.037302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.037393 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.038001 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.038016 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.039783 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 15:52:22.040052 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.034603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:50775 (local address 127.30.76.126:36367)
0808 15:52:22.034715 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.034718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.034775 (+ 57us) server_negotiation.cc:407] Connection header received
0808 15:52:22.034883 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.034886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.034918 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.034973 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.035365 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.035892 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.036991 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.037188 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.037261 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.039931 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:50775: BlockingRecv error: recv got EOF from 127.30.76.66:50775 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 15:52:22.040355 11310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:52:22.134995 11312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:22.135223 11312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:22.135264 11312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:22.139360 11312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:22.139412 11312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:22.139468 11312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:22.144579 11312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/krb5kdc/oryx_127.30.76.67.keytab
--principal=oryx/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:44461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=46519
--tserver_master_addrs=127.30.76.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:22.145574 11312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:22.146582 11312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:22.155843 11326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:22.155854 11321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:22.155854 11324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:22.156292 11312 server_base.cc:1034] running on GCE node
I20260808 15:52:22.156680 11312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:22.157236 11312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:22.158396 11312 hybrid_clock.cc:648] HybridClock initialized: now 1786204342158361 us; error 45 us; skew 500 ppm
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204342, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:22.161218 11312 init.cc:377] Logged in from keytab as oryx/127.30.76.67@KRBTEST.COM (short username oryx)
I20260808 15:52:22.162015 11312 webserver.cc:460] Webserver started at http://127.30.76.67:46519/ using document root <none> and password file <none>
I20260808 15:52:22.162529 11312 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:22.162599 11312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:22.166266 11312 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:22.168305 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:22.169520 11312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:22.169656 11312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "eec577c912aa46b8845f7b4687c4c2a5"
format_stamp: "Formatted at 2026-08-08 15:52:21 on dist-test-slave-sb2z"
server_key: "359f0b3f1d1383372b3fe4ed80d9185d"
server_key_iv: "d485cb23fe9432ed29b6bbe3501d7e99"
server_key_version: "encryptionkey@0"
I20260808 15:52:22.170173 11312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:22.190132 11312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:22.193041 11312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:22.193240 11312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:22.193817 11312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:22.194609 11312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:22.194664 11312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:22.194734 11312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:22.194774 11312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:22.203490 11312 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44461
I20260808 15:52:22.203547 11489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44461 every 8 connection(s)
I20260808 15:52:22.204515 11312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:22.212857 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 11312
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:52:22.215867 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.206322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:34771 (local address 127.30.76.126:36367)
0808 15:52:22.206463 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.206466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.207278 (+ 812us) server_negotiation.cc:407] Connection header received
0808 15:52:22.208536 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.208538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.208588 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.208667 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.209781 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.210303 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.211378 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.211667 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.211746 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.215739 (+ 3993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:34771: BlockingRecv error: recv got EOF from 127.30.76.67:34771 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260808 15:52:22.216614 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.206687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:34771)
0808 15:52:22.207112 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.207155 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.208333 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.208829 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.208836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.209181 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.209650 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.209663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.210409 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.210411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.211272 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.211281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.211795 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.211798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.211932 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.212456 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.212471 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.215651 (+ 3180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":116,"threads_started":1}
W20260808 15:52:22.217052 11490 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:36367 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.76.126:36367: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:22.222429 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.217510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:38097)
0808 15:52:22.217640 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.217656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.217734 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.217974 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.217977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.218132 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.218215 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.218219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.218916 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.218921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.219734 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.219763 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.220196 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.220199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.220287 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.220640 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.220651 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.222287 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:52:22.222517 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.217613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:38097 (local address 127.30.76.126:36367)
0808 15:52:22.217718 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.217721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.217734 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:22.217828 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.217831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.217868 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.217959 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.218325 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.218794 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.219856 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.220080 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.220194 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.222395 (+ 2201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:38097: BlockingRecv error: recv got EOF from 127.30.76.67:38097 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 15:52:22.223466 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.213876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.30.76.126:36367)
0808 15:52:22.214110 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.214136 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.214153 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:52:22.214190 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.214194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.214255 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.214398 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.215381 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.216016 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.217194 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.217515 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.217619 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.219112 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:22.219118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:22.219144 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:22.219174 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:22.221234 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:22.221727 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:22.221732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:22.221737 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:22.221859 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:22.222172 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:22.222176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:22.222178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:22.222526 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:22.222752 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:22.223089 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 15:52:22.223340 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":102,"threads_started":1}
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:22.228315 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.223165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:43213)
0808 15:52:22.223319 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.223332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.223404 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.223728 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.223731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.223873 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.223933 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.223937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.224775 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.224777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.225470 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.225481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.225993 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.225997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.226069 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.226562 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.226573 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.228184 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260808 15:52:22.228392 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.223225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:43213 (local address 127.30.76.126:36367)
0808 15:52:22.223353 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.223360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.223374 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:52:22.223504 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.223507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.223564 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.223633 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.224039 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.224664 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.225585 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.225867 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.225957 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.228286 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:43213: BlockingRecv error: recv got EOF from 127.30.76.67:43213 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 15:52:22.228576 11490 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:22 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:22.845367 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:22.840261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:58051)
0808 15:52:22.840547 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:22.840566 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:22.840674 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:22.840901 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:22.840903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:22.841094 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:22.841194 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.841199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.841917 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.841919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.842476 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:22.842483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.842898 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.842901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:22.842961 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.843450 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:22.843461 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:22.845248 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
W20260808 15:52:22.845522 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:22.840359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:58051 (local address 127.30.76.126:36367)
0808 15:52:22.840666 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:22.840670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:22.840683 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:22.840741 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:22.840743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:22.840786 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:22.840862 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:22.841295 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.841825 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.842573 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:22.842795 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:22.842870 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:22.845382 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:58051: BlockingRecv error: recv got EOF from 127.30.76.65:58051 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":113,"threads_started":1}
Aug 08 15:52:23 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:23 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:23.046917 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:23.040988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:42577)
0808 15:52:23.041265 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:23.041287 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:23.041425 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:23.041712 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:23.041715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:23.041927 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:23.042020 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:23.042025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.042820 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.042826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:23.043393 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:23.043402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.043915 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.043919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:23.044005 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:23.044615 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:23.044627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:23.046791 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":85,"threads_started":1}
W20260808 15:52:23.046988 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:23.041054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:42577 (local address 127.30.76.126:36367)
0808 15:52:23.041234 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:23.041238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:23.041364 (+ 126us) server_negotiation.cc:407] Connection header received
0808 15:52:23.041526 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:23.041529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:23.041580 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:23.041686 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:23.042172 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.042704 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.043520 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.043799 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.043911 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:23.046895 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:42577: BlockingRecv error: recv got EOF from 127.30.76.66:42577 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 15:52:23 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:23 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:23.234860 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:23.229301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:39827)
0808 15:52:23.229576 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:23.229590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:23.229710 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:23.230010 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:23.230014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:23.230264 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:23.230413 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:23.230419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.231196 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.231203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:23.231821 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:23.231830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.232289 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.232293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:23.232384 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:23.232867 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:23.232878 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:23.234743 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
W20260808 15:52:23.235030 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:23.229428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:39827 (local address 127.30.76.126:36367)
0808 15:52:23.229598 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:23.229601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:23.229700 (+ 99us) server_negotiation.cc:407] Connection header received
0808 15:52:23.229798 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:23.229801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:23.229861 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:23.229986 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:23.230535 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.231072 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.231947 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.232174 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.232287 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:23.234875 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:39827: BlockingRecv error: recv got EOF from 127.30.76.67:39827 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 15:52:23 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:23 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:23.851706 11504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:23.846306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:45125)
0808 15:52:23.846597 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:23.846610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:23.846699 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:23.846926 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:23.846928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:23.847108 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:23.847216 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:23.847222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.848049 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.848052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:23.848607 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:23.848615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.849096 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.849099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:23.849172 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:23.849672 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:23.849687 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:23.851490 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
W20260808 15:52:23.851845 11505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:23.846395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:45125 (local address 127.30.76.126:36367)
0808 15:52:23.846705 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:23.846708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:23.846723 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:23.846759 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:23.846761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:23.846800 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:23.846884 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:23.847352 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.847954 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.848728 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:23.848980 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:23.849067 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:23.851683 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:45125: BlockingRecv error: recv got EOF from 127.30.76.65:45125 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":151,"threads_started":1}
Aug 08 15:52:24 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:24 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:24.058372 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:24.047810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:57937)
0808 15:52:24.048262 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:24.048276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:24.048425 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:24.048704 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:24.048707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:24.048917 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:24.049008 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:24.049013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.049784 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.049789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:24.050344 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:24.050353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.050842 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.050846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:24.050941 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:24.056071 (+ 5130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:24.056084 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:24.058176 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":98,"threads_started":1}
W20260808 15:52:24.058411 11505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:24.048008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:57937 (local address 127.30.76.126:36367)
0808 15:52:24.048157 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:24.048160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:24.048372 (+ 212us) server_negotiation.cc:407] Connection header received
0808 15:52:24.048529 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:24.048533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:24.048586 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:24.048676 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:24.049133 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.049671 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.050481 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.050707 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.050826 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:24.058304 (+ 7478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:57937: BlockingRecv error: recv got EOF from 127.30.76.66:57937 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 15:52:24 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:24 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:24.241194 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:24.235585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:60349)
0808 15:52:24.235927 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:24.235944 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:24.236059 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:24.236351 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:24.236354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:24.236568 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:24.236666 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:24.236672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.237424 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.237430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:24.237997 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:24.238004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.238477 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.238480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:24.238562 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:24.239159 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:24.239171 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:24.241023 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":129,"threads_started":1}
W20260808 15:52:24.241261 11505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:24.235782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:60349 (local address 127.30.76.126:36367)
0808 15:52:24.235928 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:24.235934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:24.236001 (+ 67us) server_negotiation.cc:407] Connection header received
0808 15:52:24.236179 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:24.236182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:24.236238 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:24.236328 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:24.236787 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.237313 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.238120 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.238363 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.238473 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:24.241152 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:60349: BlockingRecv error: recv got EOF from 127.30.76.67:60349 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 15:52:24 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:24 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:24.858522 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:24.852820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:36183)
0808 15:52:24.853090 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:24.853104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:24.853216 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:24.853499 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:24.853501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:24.853721 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:24.853855 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:24.853865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.854958 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.854962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:24.855670 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:24.855681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.856120 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.856122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:24.856187 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:24.856662 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:24.856673 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:24.858409 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
W20260808 15:52:24.858718 11509 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:24.852869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:36183 (local address 127.30.76.126:36367)
0808 15:52:24.853114 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:24.853118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:24.853165 (+ 47us) server_negotiation.cc:407] Connection header received
0808 15:52:24.853298 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:24.853300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:24.853345 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:24.853476 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:24.853986 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.854836 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.855782 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:24.856017 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:24.856094 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:24.858539 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:36183: BlockingRecv error: recv got EOF from 127.30.76.65:36183 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":98,"threads_started":1}
Aug 08 15:52:25 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:25 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:25.065274 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:25.059143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:56031)
0808 15:52:25.059483 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:25.059500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:25.059689 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:25.060028 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:25.060031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:25.060271 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:25.060424 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:25.060436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.061312 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.061314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:25.061865 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:25.061872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.062280 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.062283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:25.062346 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:25.062843 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:25.062855 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:25.065155 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":92,"threads_started":1}
W20260808 15:52:25.065421 11509 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:25.059282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56031 (local address 127.30.76.126:36367)
0808 15:52:25.059462 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:25.059467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:25.059611 (+ 144us) server_negotiation.cc:407] Connection header received
0808 15:52:25.059807 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:25.059809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:25.059889 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:25.059983 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:25.060573 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.061205 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.061966 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.062184 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.062258 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:25.065294 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:56031: BlockingRecv error: recv got EOF from 127.30.76.66:56031 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 15:52:25 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:25 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:25.247594 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:25.242047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:51819)
0808 15:52:25.242333 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:25.242347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:25.242449 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:25.242696 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:25.242698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:25.242875 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:25.242971 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:25.242976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.243871 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.243877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:25.244453 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:25.244461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.244939 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.244943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:25.245034 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:25.245526 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:25.245538 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:25.247410 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
W20260808 15:52:25.247725 11509 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:25.242144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:51819 (local address 127.30.76.126:36367)
0808 15:52:25.242291 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:25.242296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:25.242414 (+ 118us) server_negotiation.cc:407] Connection header received
0808 15:52:25.242527 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:25.242529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:25.242581 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:25.242661 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:25.243102 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.243756 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.244578 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.244816 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.244925 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:25.247603 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:51819: BlockingRecv error: recv got EOF from 127.30.76.67:51819 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 15:52:25 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:25 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:25.864483 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:25.859299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:36901)
0808 15:52:25.859621 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:25.859646 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:25.859737 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:25.859969 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:25.859971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:25.860155 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:25.860250 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:25.860256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.860987 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.860990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:25.861525 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:25.861532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.861937 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.861939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:25.861998 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:25.862471 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:25.862482 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:25.864368 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":90,"threads_started":1}
W20260808 15:52:25.864642 11517 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:25.859383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:36901 (local address 127.30.76.126:36367)
0808 15:52:25.859700 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:25.859703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:25.859717 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:52:25.859816 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:25.859818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:25.859861 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:25.859929 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:25.860352 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.860883 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.861619 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:25.861840 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:25.861914 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:25.864500 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:36901: BlockingRecv error: recv got EOF from 127.30.76.65:36901 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
Aug 08 15:52:26 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:26 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:26.071877 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:26.066184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:47917)
0808 15:52:26.066507 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:26.066520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:26.066661 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:26.066905 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:26.066907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:26.067090 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:26.067186 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:26.067191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.067986 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.067989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:26.068523 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:26.068530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.069016 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.069018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:26.069078 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:26.069587 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:26.069598 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:26.071688 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":116,"threads_started":1}
W20260808 15:52:26.072000 11517 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:26.066221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:47917 (local address 127.30.76.126:36367)
0808 15:52:26.066369 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:26.066374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:26.066625 (+ 251us) server_negotiation.cc:407] Connection header received
0808 15:52:26.066748 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:26.066750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:26.066804 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:26.066871 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:26.067270 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.067893 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.068634 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.068877 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.069013 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:26.071827 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:47917: BlockingRecv error: recv got EOF from 127.30.76.66:47917 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 15:52:26 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:26 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:26.253793 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:26.248344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:37953)
0808 15:52:26.248616 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:26.248629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:26.248773 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:26.249053 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:26.249055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:26.249274 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:26.249368 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:26.249373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.250158 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.250162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:26.250716 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:26.250724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.251184 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.251188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:26.251274 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:26.251797 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:26.251809 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:26.253603 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
W20260808 15:52:26.254041 11517 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:26.248386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37953 (local address 127.30.76.126:36367)
0808 15:52:26.248536 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:26.248544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:26.248715 (+ 171us) server_negotiation.cc:407] Connection header received
0808 15:52:26.248874 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:26.248878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:26.248938 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:26.249026 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:26.249497 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.250036 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.250839 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.251067 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.251180 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:26.253805 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:37953: BlockingRecv error: recv got EOF from 127.30.76.67:37953 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 15:52:26 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:26 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:26.871052 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:26.865380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:39823)
0808 15:52:26.865674 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:26.865687 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:26.865779 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:26.866026 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:26.866028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:26.866216 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:26.866315 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:26.866320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.867060 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.867062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:26.867809 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:26.867820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.868401 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.868405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:26.868487 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:26.869060 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:26.869072 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:26.870934 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
W20260808 15:52:26.871207 11523 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:26.865449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39823 (local address 127.30.76.126:36367)
0808 15:52:26.865699 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:26.865705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:26.865741 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:52:26.865867 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:26.865869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:26.865914 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:26.865991 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:26.866434 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.866968 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.867930 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:26.868260 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:26.868375 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:26.871066 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:39823: BlockingRecv error: recv got EOF from 127.30.76.65:39823 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":64,"threads_started":1}
Aug 08 15:52:27 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:27 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:27.078541 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:27.072814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:56443)
0808 15:52:27.073169 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:27.073186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:27.073315 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:27.073622 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:27.073627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:27.073884 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:27.074024 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:27.074036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.074951 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.074954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:27.075684 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:27.075693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.076093 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.076096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:27.076154 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:27.076647 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:27.076659 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:27.078423 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
W20260808 15:52:27.078680 11523 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:27.073014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56443 (local address 127.30.76.126:36367)
0808 15:52:27.073165 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:27.073170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:27.073281 (+ 111us) server_negotiation.cc:407] Connection header received
0808 15:52:27.073424 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:27.073428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:27.073489 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:27.073575 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:27.074135 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.074830 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.075789 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.075996 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.076075 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:27.078556 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:56443: BlockingRecv error: recv got EOF from 127.30.76.66:56443 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 15:52:27 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:27 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:27.260399 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:27.254575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:37497)
0808 15:52:27.254870 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:27.254884 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:27.255042 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:27.255334 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:27.255336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:27.255543 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:27.255637 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:27.255643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.256425 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.256427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:27.256984 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:27.256991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.257503 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.257507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:27.257598 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:27.258206 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:27.258225 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:27.260222 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":107,"threads_started":1}
W20260808 15:52:27.260447 11523 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:27.254753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37497 (local address 127.30.76.126:36367)
0808 15:52:27.254885 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:27.254889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:27.254933 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:52:27.255171 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:27.255173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:27.255223 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:27.255294 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:27.255736 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.256330 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.257081 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.257360 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.257481 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:27.260345 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:37497: BlockingRecv error: recv got EOF from 127.30.76.67:37497 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 15:52:27 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:27 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:27.877887 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:27.871938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:41569)
0808 15:52:27.872257 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:27.872273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:27.872369 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:27.872656 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:27.872659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:27.872852 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:27.872983 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:27.872989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.873834 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.873837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:27.874830 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:27.874844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.875341 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.875344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:27.875405 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:27.875938 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:27.875950 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:27.877771 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":129,"threads_started":1}
W20260808 15:52:27.878039 11529 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:27.872087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:41569 (local address 127.30.76.126:36367)
0808 15:52:27.872333 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:27.872336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:27.872349 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:27.872451 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:27.872453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:27.872496 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:27.872568 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:27.873113 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.873720 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.874942 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:27.875240 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:27.875321 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:27.877904 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:41569: BlockingRecv error: recv got EOF from 127.30.76.65:41569 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
Aug 08 15:52:28 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:28 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:28.084815 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:28.079385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:44675)
0808 15:52:28.079684 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:28.079696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:28.079793 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:28.079999 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:28.080001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:28.080191 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:28.080303 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:28.080308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.081144 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.081147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:28.081863 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:28.081874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.082284 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.082287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:28.082351 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:28.082820 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:28.082831 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:28.084701 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
W20260808 15:52:28.085000 11529 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:28.079461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:44675 (local address 127.30.76.126:36367)
0808 15:52:28.079661 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:28.079665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:28.079760 (+ 95us) server_negotiation.cc:407] Connection header received
0808 15:52:28.079867 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:28.079869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:28.079913 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:28.079978 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:28.080413 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.081040 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.081964 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.082187 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.082266 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:28.084832 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:44675: BlockingRecv error: recv got EOF from 127.30.76.66:44675 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 15:52:28 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:28 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:28.266441 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:28.261317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:51771)
0808 15:52:28.261591 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:28.261604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:28.261722 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:28.261941 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:28.261944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:28.262128 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:28.262221 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:28.262226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.262946 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.262948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:28.263483 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:28.263490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.263925 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.263928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:28.263991 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:28.264474 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:28.264485 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:28.266307 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
W20260808 15:52:28.266594 11529 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:28.261383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:51771 (local address 127.30.76.126:36367)
0808 15:52:28.261517 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:28.261521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:28.261689 (+ 168us) server_negotiation.cc:407] Connection header received
0808 15:52:28.261788 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:28.261790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:28.261836 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:28.261905 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:28.262330 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.262848 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.263609 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.263829 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.263905 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:28.266472 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:51771: BlockingRecv error: recv got EOF from 127.30.76.67:51771 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:52:28 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:28 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:28.885015 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:28.878897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:45023)
0808 15:52:28.879318 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:28.879335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:28.879467 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:28.879785 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:28.879788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:28.879977 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:28.880072 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:28.880077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.880968 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.880971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:28.881765 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:28.881775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.882250 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.882254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:28.882343 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:28.882862 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:28.882874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:28.884843 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":109,"threads_started":1}
W20260808 15:52:28.885080 11534 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:28.878981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:45023 (local address 127.30.76.126:36367)
0808 15:52:28.879257 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:28.879262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:28.879430 (+ 168us) server_negotiation.cc:407] Connection header received
0808 15:52:28.879600 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:28.879604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:28.879662 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:28.879748 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:28.880181 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.880860 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.881874 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:28.882129 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:28.882251 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:28.884961 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:45023: BlockingRecv error: recv got EOF from 127.30.76.65:45023 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
Aug 08 15:52:29 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:29 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:29.093277 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:29.086104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:59385)
0808 15:52:29.086857 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:29.086872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:29.086980 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:29.087279 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:29.087282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:29.087468 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:29.087642 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:29.087654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.088620 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.088624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:29.089387 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:29.089401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.090032 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.090036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:29.090120 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:29.090727 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:29.090746 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:29.093093 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":529,"threads_started":1}
W20260808 15:52:29.093348 11534 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:29.086169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:59385 (local address 127.30.76.126:36367)
0808 15:52:29.086318 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:29.086324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:29.086964 (+ 640us) server_negotiation.cc:407] Connection header received
0808 15:52:29.087053 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:29.087059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:29.087134 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:29.087266 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:29.087785 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.088507 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.089516 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.089892 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.090009 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:29.093230 (+ 3221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:59385: BlockingRecv error: recv got EOF from 127.30.76.66:59385 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:52:29 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:29 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:29.273032 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:29.267425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:56105)
0808 15:52:29.267777 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:29.267790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:29.267893 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:29.268117 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:29.268119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:29.268309 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:29.268401 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:29.268406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.269103 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.269105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:29.269665 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:29.269673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.270061 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.270064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:29.270123 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:29.270625 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:29.270638 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:29.272858 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":136,"threads_started":1}
W20260808 15:52:29.273089 11534 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:29.267490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:56105 (local address 127.30.76.126:36367)
0808 15:52:29.267681 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:29.267684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:29.267892 (+ 208us) server_negotiation.cc:407] Connection header received
0808 15:52:29.267958 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:29.267960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:29.268009 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:29.268081 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:29.268500 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.269015 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.269760 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.269969 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.270041 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:29.272996 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:56105: BlockingRecv error: recv got EOF from 127.30.76.67:56105 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 08 15:52:29 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:29 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:29.891459 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:29.885834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:36301)
0808 15:52:29.886132 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:29.886146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:29.886236 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:29.886472 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:29.886474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:29.886668 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:29.886766 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:29.886772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.887479 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.887481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:29.888096 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:29.888106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.888690 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.888693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:29.888784 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:29.889397 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:29.889411 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:29.891342 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":111,"threads_started":1}
W20260808 15:52:29.891692 11540 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:29.885992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:36301 (local address 127.30.76.126:36367)
0808 15:52:29.886209 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:29.886212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:29.886225 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:29.886314 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:29.886316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:29.886357 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:29.886434 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:29.886870 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.887388 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.888252 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:29.888560 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:29.888677 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:29.891484 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:36301: BlockingRecv error: recv got EOF from 127.30.76.65:36301 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
Aug 08 15:52:30 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:30 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:30.100823 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:30.094193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:38329)
0808 15:52:30.094470 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:30.094483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:30.094581 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:30.094860 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:30.094863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:30.095056 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:30.095161 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:30.095166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.096567 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.096570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:30.097522 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:30.097534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.098151 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.098155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:30.098238 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:30.098778 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:30.098790 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:30.100702 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
W20260808 15:52:30.101012 11540 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:30.094245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:38329 (local address 127.30.76.126:36367)
0808 15:52:30.094424 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:30.094428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:30.094577 (+ 149us) server_negotiation.cc:407] Connection header received
0808 15:52:30.094656 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:30.094659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:30.094723 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:30.094821 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:30.095270 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.096407 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.097679 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.098032 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.098155 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:30.100839 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:38329: BlockingRecv error: recv got EOF from 127.30.76.66:38329 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:52:30 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:30 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:30.279517 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:30.274051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:35503)
0808 15:52:30.274338 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:30.274351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:30.274452 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:30.274671 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:30.274673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:30.274848 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:30.274989 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:30.274999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.275800 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.275803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:30.276347 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:30.276355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.276755 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.276758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:30.276815 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:30.277280 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:30.277291 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:30.279347 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":121,"threads_started":1}
W20260808 15:52:30.279601 11540 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:30.274091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:35503 (local address 127.30.76.126:36367)
0808 15:52:30.274238 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:30.274243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:30.274424 (+ 181us) server_negotiation.cc:407] Connection header received
0808 15:52:30.274516 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:30.274518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:30.274564 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:30.274630 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:30.275101 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.275702 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.276445 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.276661 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.276738 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:30.279465 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:35503: BlockingRecv error: recv got EOF from 127.30.76.67:35503 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:52:30 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:30 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:30.897747 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:30.892411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:51101)
0808 15:52:30.892699 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:30.892713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:30.892805 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:30.893035 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:30.893037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:30.893223 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:30.893323 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:30.893329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.894050 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.894052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:30.894589 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:30.894596 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.895024 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.895027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:30.895087 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:30.895654 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:30.895666 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:30.897534 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
W20260808 15:52:30.897806 11546 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:30.892594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:51101 (local address 127.30.76.126:36367)
0808 15:52:30.892823 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:30.892827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:30.892839 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:52:30.892873 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:30.892875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:30.892915 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:30.892996 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:30.893424 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.893958 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.894706 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:30.894924 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:30.895000 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:30.897670 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:51101: BlockingRecv error: recv got EOF from 127.30.76.65:51101 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
Aug 08 15:52:31 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:31 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:31.107142 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:31.101690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:52777)
0808 15:52:31.101957 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:31.101972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:31.102079 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:31.102297 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:31.102299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:31.102476 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:31.102569 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:31.102574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.103293 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.103296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:31.103940 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:31.103950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.104358 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.104361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:31.104421 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:31.104963 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:31.104975 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:31.106919 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
W20260808 15:52:31.107154 11546 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:31.101736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:52777 (local address 127.30.76.126:36367)
0808 15:52:31.101897 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:31.101902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:31.102054 (+ 152us) server_negotiation.cc:407] Connection header received
0808 15:52:31.102174 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:31.102176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:31.102225 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:31.102284 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:31.102674 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.103200 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.104041 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.104260 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.104340 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:31.107043 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:52777: BlockingRecv error: recv got EOF from 127.30.76.66:52777 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 15:52:31 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:31 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:31.285817 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:31.280424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:46485)
0808 15:52:31.280721 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:31.280734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:31.280867 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:31.281182 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:31.281184 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:31.281421 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:31.281521 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:31.281528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.282292 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.282295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:31.282841 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:31.282849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.283267 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.283269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:31.283325 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:31.283803 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:31.283814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:31.285703 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":87,"threads_started":1}
W20260808 15:52:31.285955 11546 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:31.280493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:46485 (local address 127.30.76.126:36367)
0808 15:52:31.280698 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:31.280702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:31.280818 (+ 116us) server_negotiation.cc:407] Connection header received
0808 15:52:31.280989 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:31.280995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:31.281050 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:31.281157 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:31.281652 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.282196 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.282943 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.283170 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.283248 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:31.285837 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:46485: BlockingRecv error: recv got EOF from 127.30.76.67:46485 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 15:52:31 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:31 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:31.904548 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:31.898659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:49535)
0808 15:52:31.898981 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:31.898996 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:31.899087 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:31.899345 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:31.899347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:31.899559 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:31.899657 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:31.899663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.900432 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.900438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:31.901091 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:31.901101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.901720 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.901724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:31.901812 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:31.902354 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:31.902373 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:31.904432 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":73,"threads_started":1}
W20260808 15:52:31.904835 11551 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:31.898703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:49535 (local address 127.30.76.126:36367)
0808 15:52:31.899019 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:31.899022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:31.899038 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:31.899181 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:31.899183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:31.899227 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:31.899302 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:31.899759 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.900294 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.901224 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:31.901586 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:31.901714 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:31.904569 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:49535: BlockingRecv error: recv got EOF from 127.30.76.65:49535 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":135,"threads_started":1}
Aug 08 15:52:32 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:32 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:32.113297 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:32.108105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:45301)
0808 15:52:32.108372 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:32.108386 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:32.108520 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:32.108798 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:32.108800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:32.108979 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:32.109076 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:32.109081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.109886 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.109888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:32.110441 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:32.110449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.110853 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.110856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:32.110918 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:32.111385 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:32.111396 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:32.113182 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
W20260808 15:52:32.113473 11551 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:32.108155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:45301 (local address 127.30.76.126:36367)
0808 15:52:32.108307 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:32.108311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:32.108465 (+ 154us) server_negotiation.cc:407] Connection header received
0808 15:52:32.108638 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:32.108641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:32.108696 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:32.108766 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:32.109200 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.109789 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.110538 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.110758 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.110832 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:32.113318 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:45301: BlockingRecv error: recv got EOF from 127.30.76.66:45301 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 15:52:32 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:32 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:32.291877 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:32.286589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:51795)
0808 15:52:32.286847 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:32.286860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:32.286944 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:32.287177 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:32.287179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:32.287357 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:32.287451 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:32.287456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.288279 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.288281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:32.288836 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:32.288844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.289302 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.289304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:32.289361 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:32.289826 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:32.289837 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:32.291740 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":82,"threads_started":1}
W20260808 15:52:32.291998 11551 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:32.286730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:51795 (local address 127.30.76.126:36367)
0808 15:52:32.286896 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:32.286900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:32.286915 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:52:32.287027 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:32.287029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:32.287075 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:32.287139 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:32.287611 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.288178 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.288964 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.289201 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.289288 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:32.291878 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:51795: BlockingRecv error: recv got EOF from 127.30.76.67:51795 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 15:52:32 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:32 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:32.910758 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:32.905430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:42835)
0808 15:52:32.905719 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:32.905733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:32.905826 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:32.906077 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:32.906079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:32.906277 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:32.906375 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:32.906380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.907102 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.907104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:32.907681 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:32.907689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.908095 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.908098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:32.908167 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:32.908746 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:32.908759 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:32.910615 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":69,"threads_started":1}
W20260808 15:52:32.910940 11555 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:32.905486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:42835 (local address 127.30.76.126:36367)
0808 15:52:32.905776 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:32.905780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:32.905794 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:52:32.905913 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:32.905915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:32.905960 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:32.906044 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:32.906476 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.907009 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.907779 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:32.907996 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:32.908073 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:32.910765 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:42835: BlockingRecv error: recv got EOF from 127.30.76.65:42835 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":98,"threads_started":1}
Aug 08 15:52:33 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:33 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:33.119278 11557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:33.114078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:41225)
0808 15:52:33.114372 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:33.114386 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:33.114485 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:33.114706 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:33.114708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:33.114885 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:33.114977 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:33.114982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.115770 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.115773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:33.116316 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:33.116323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.116722 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.116725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:33.116786 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:33.117257 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:33.117268 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:33.119096 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
W20260808 15:52:33.119319 11555 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:33.114118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:41225 (local address 127.30.76.126:36367)
0808 15:52:33.114273 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:33.114278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:33.114453 (+ 175us) server_negotiation.cc:407] Connection header received
0808 15:52:33.114548 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:33.114550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:33.114596 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:33.114669 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:33.115069 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.115675 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.116409 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.116619 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.116703 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:33.119220 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:41225: BlockingRecv error: recv got EOF from 127.30.76.66:41225 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 15:52:33 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:33 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:33.298302 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:33.292781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:56521)
0808 15:52:33.293159 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:33.293183 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:33.293313 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:33.293637 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:33.293641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:33.293890 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:33.294025 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:33.294032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.294869 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.294871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:33.295433 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:33.295441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.295877 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.295879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:33.295939 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:33.296428 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:33.296439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:33.298187 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
W20260808 15:52:33.298445 11555 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:33.292988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:56521 (local address 127.30.76.126:36367)
0808 15:52:33.293157 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:33.293163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:33.293239 (+ 76us) server_negotiation.cc:407] Connection header received
0808 15:52:33.293429 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:33.293435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:33.293499 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:33.293575 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:33.294220 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.294772 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.295558 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.295780 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.295856 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:33.298324 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:56521: BlockingRecv error: recv got EOF from 127.30.76.67:56521 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:52:33 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:33 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:33.917734 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:33.911916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:35625)
0808 15:52:33.912250 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:33.912264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:33.912357 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:33.912578 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:33.912580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:33.912759 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:33.912852 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:33.912857 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.913573 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.913576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:33.914165 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:33.914173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.914710 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.914714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:33.914818 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:33.915485 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:33.915500 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:33.917534 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":108,"threads_started":1}
W20260808 15:52:33.918080 11561 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:33.912025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:35625 (local address 127.30.76.126:36367)
0808 15:52:33.912288 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:33.912292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:33.912308 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:52:33.912444 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:33.912446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:33.912491 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:33.912569 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:33.912943 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.913479 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.914267 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:33.914554 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:33.914666 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:33.917793 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:35625: BlockingRecv error: recv got EOF from 127.30.76.65:35625 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
Aug 08 15:52:34 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:34 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:34.128660 11562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:34.120157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:57085)
0808 15:52:34.120432 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:34.120454 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:34.120553 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:34.120793 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:34.120795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:34.120987 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:34.121084 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:34.121089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.121998 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.122001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:34.122563 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:34.122572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.123064 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.123068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:34.123168 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:34.126602 (+ 3434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:34.126615 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:34.128488 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
W20260808 15:52:34.128726 11561 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:34.120231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:57085 (local address 127.30.76.126:36367)
0808 15:52:34.120391 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:34.120396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:34.120520 (+ 124us) server_negotiation.cc:407] Connection header received
0808 15:52:34.120638 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:34.120640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:34.120685 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:34.120756 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:34.121183 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.121882 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.122690 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.122930 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.123060 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:34.128609 (+ 5549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:57085: BlockingRecv error: recv got EOF from 127.30.76.66:57085 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 15:52:34 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:34 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:34.304409 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:34.299228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:53763)
0808 15:52:34.299494 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:34.299508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:34.299617 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:34.299862 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:34.299864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:34.300043 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:34.300145 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:34.300150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.300841 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.300843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:34.301397 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:34.301405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.301792 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.301795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:34.301853 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:34.302323 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:34.302352 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:34.304294 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
W20260808 15:52:34.304581 11561 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:34.299351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:53763 (local address 127.30.76.126:36367)
0808 15:52:34.299520 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:34.299574 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:34.299591 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:52:34.299705 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:34.299707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:34.299756 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:34.299824 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:34.300240 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.300752 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.301492 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.301700 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.301774 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:34.304424 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:53763: BlockingRecv error: recv got EOF from 127.30.76.67:53763 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 15:52:34 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:34 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:34.924495 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:34.918724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:42967)
0808 15:52:34.919024 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:34.919040 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:34.919158 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:34.919597 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:34.919600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:34.919794 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:34.919894 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:34.919900 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.920688 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.920691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:34.921311 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:34.921320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.921741 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.921744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:34.921811 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:34.922363 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:34.922376 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:34.924373 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":113,"threads_started":1}
W20260808 15:52:34.924687 11565 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:34.918743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:42967 (local address 127.30.76.126:36367)
0808 15:52:34.919002 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:34.919005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:34.919092 (+ 87us) server_negotiation.cc:407] Connection header received
0808 15:52:34.919237 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:34.919239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:34.919284 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:34.919352 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:34.920041 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.920576 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.921418 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:34.921634 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:34.921709 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:34.924515 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:42967: BlockingRecv error: recv got EOF from 127.30.76.65:42967 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":98,"threads_started":1}
Aug 08 15:52:35 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:35 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:35.134722 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:35.129564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:41313)
0808 15:52:35.129877 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:35.129890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:35.129989 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:35.130216 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:35.130219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:35.130397 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:35.130491 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:35.130496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.131211 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.131213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:35.131788 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:35.131795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.132203 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.132205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:35.132266 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:35.132751 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:35.132763 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:35.134597 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":133,"threads_started":1}
W20260808 15:52:35.134861 11565 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:35.129683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:41313 (local address 127.30.76.126:36367)
0808 15:52:35.129840 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:35.129845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:35.129942 (+ 97us) server_negotiation.cc:407] Connection header received
0808 15:52:35.130066 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:35.130068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:35.130113 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:35.130175 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:35.130592 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.131118 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.131888 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.132103 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.132182 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:35.134746 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:41313: BlockingRecv error: recv got EOF from 127.30.76.66:41313 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 15:52:35 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:35 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:35.311259 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:35.305179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:48635)
0808 15:52:35.305476 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:35.305490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:35.305591 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:35.305786 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:35.305788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:35.305972 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:35.306062 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:35.306066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.306780 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.306782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:35.307312 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:35.307320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.307816 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.307818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:35.307892 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:35.308873 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:35.308894 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:35.311141 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":125,"threads_started":1}
W20260808 15:52:35.311426 11565 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:35.305228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:48635 (local address 127.30.76.126:36367)
0808 15:52:35.305390 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:35.305395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:35.305548 (+ 153us) server_negotiation.cc:407] Connection header received
0808 15:52:35.305663 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:35.305665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:35.305711 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:35.305775 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:35.306159 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.306687 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.307420 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.307717 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.307800 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:35.311273 (+ 3473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:48635: BlockingRecv error: recv got EOF from 127.30.76.67:48635 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 15:52:35 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:35 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:35.932091 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:35.925475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:56907)
0808 15:52:35.925842 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:35.925879 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:35.926000 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:35.926356 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:35.926359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:35.926670 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:35.926859 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:35.926870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.927811 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.927816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:35.928455 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:35.928464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.929015 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.929019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:35.929143 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:35.929783 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:35.929797 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:35.931851 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
W20260808 15:52:35.932117 11570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:35.925543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:56907 (local address 127.30.76.126:36367)
0808 15:52:35.925933 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:35.925938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:35.925955 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:52:35.926133 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:35.926136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:35.926205 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:35.926318 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:35.926998 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.927688 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.928591 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:35.928861 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:35.928986 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:35.931988 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:56907: BlockingRecv error: recv got EOF from 127.30.76.65:56907 (error 108)
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":77,"threads_started":1}
Aug 08 15:52:36 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:36 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:36.141036 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:36.135796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:57203)
0808 15:52:36.136069 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:36.136085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:36.136202 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:36.136446 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:36.136449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:36.136631 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:36.136726 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:36.136731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.137505 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.137508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:36.138064 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:36.138071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.138487 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.138490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:36.138551 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:36.139037 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:36.139048 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:36.140882 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
W20260808 15:52:36.141227 11570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:36.135919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:57203 (local address 127.30.76.126:36367)
0808 15:52:36.136090 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:36.136093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:36.136152 (+ 59us) server_negotiation.cc:407] Connection header received
0808 15:52:36.136286 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:36.136288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:36.136333 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:36.136403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:36.136839 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.137402 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.138170 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.138390 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.138470 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:36.141067 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:57203: BlockingRecv error: recv got EOF from 127.30.76.66:57203 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 15:52:36 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:36 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.67@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:36.317466 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:36.312195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.67:58379)
0808 15:52:36.312495 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:36.312508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:36.312607 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:36.312841 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:36.312843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:36.313021 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:36.313113 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:36.313118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.313834 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.313837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:36.314377 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:36.314384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.314783 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.314785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:36.314845 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:36.315320 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:36.315331 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:36.317350 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":119,"threads_started":1}
W20260808 15:52:36.317646 11570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:36.312238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:58379 (local address 127.30.76.126:36367)
0808 15:52:36.312388 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:36.312393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:36.312581 (+ 188us) server_negotiation.cc:407] Connection header received
0808 15:52:36.312677 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:36.312680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:36.312738 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:36.312803 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:36.313220 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.313745 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.314475 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.314691 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.314765 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:36.317480 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.67:58379: BlockingRecv error: recv got EOF from 127.30.76.67:58379 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 15:52:36 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:36 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.65@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:36.938627 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:36.933060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.65:39029)
0808 15:52:36.933355 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:36.933369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:36.933454 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:36.933691 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:36.933693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:36.933881 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:36.933977 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:36.933982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.934702 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.934705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:36.935244 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:36.935250 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.935879 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.935884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:36.935990 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:36.936628 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:36.936666 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:36.938454 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":114,"threads_started":1}
W20260808 15:52:36.938710 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:36.933169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39029 (local address 127.30.76.126:36367)
0808 15:52:36.933421 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:36.933425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:36.933438 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:36.933522 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:36.933524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:36.933564 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:36.933657 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:36.934077 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.934595 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.935370 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:36.935721 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:36.935856 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:36.938584 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.65:39029: BlockingRecv error: recv got EOF from 127.30.76.65:39029 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
Aug 08 15:52:37 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:52:37 dist-test-slave-sb2z krb5kdc[10154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.76.66@KRBTEST.COM for oryx/127.30.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:52:37.147787 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:37.142003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36367 (local address 127.30.76.66:37101)
0808 15:52:37.142292 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:37.142305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:37.142401 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:37.142610 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:37.142612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:37.142798 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:37.142889 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:37.142894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.143812 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.143816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:37.144520 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:37.144534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.145057 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.145060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:37.145147 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.145633 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:37.145645 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:37.147670 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.76.126:36367: Server oryx/127.30.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":117,"threads_started":1}
W20260808 15:52:37.147974 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:37.142071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:37101 (local address 127.30.76.126:36367)
0808 15:52:37.142229 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:37.142232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:37.142412 (+ 180us) server_negotiation.cc:407] Connection header received
0808 15:52:37.142472 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:37.142474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:37.142522 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:37.142599 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:37.143003 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.143687 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.144658 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.144929 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.145040 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.147805 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.76.66:37101: BlockingRecv error: recv got EOF from 127.30.76.66:37101 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:52:37.223850 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 10938
I20260808 15:52:37.235726 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 11147
I20260808 15:52:37.258370 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 11312
I20260808 15:52:37.266968 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 10841
2026-08-08T15:52:37Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18246 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T15:52:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:37Z Disabled control of system clock
I20260808 15:52:37.300886 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:46701
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:46701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 15:52:37.413702 11581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:37.413968 11581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:37.414002 11581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:37.418073 11581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:37.418130 11581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:37.418145 11581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:37.418162 11581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:37.418177 11581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:37.422837 11581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:46701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:46701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:37.423902 11581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:37.425283 11581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:37.434289 11586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:37.434304 11587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:37.434391 11581 server_base.cc:1034] running on GCE node
W20260808 15:52:37.434304 11592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:37.435205 11581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:37.436447 11581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:37.437614 11581 hybrid_clock.cc:648] HybridClock initialized: now 1786204357437582 us; error 48 us; skew 500 ppm
I20260808 15:52:37.439674 11581 webserver.cc:460] Webserver started at http://127.30.76.126:42639/ using document root <none> and password file <none>
I20260808 15:52:37.440296 11581 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:37.440363 11581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:37.440615 11581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:37.442719 11581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "13d3381d7f5b4c08731e6274ff61df81"
server_key_iv: "405dab205b65629a6a8e96f17553067f"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.443367 11581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "13d3381d7f5b4c08731e6274ff61df81"
server_key_iv: "405dab205b65629a6a8e96f17553067f"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.448650 11581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260808 15:52:37.451198 11599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.452416 11581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:52:37.452564 11581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "13d3381d7f5b4c08731e6274ff61df81"
server_key_iv: "405dab205b65629a6a8e96f17553067f"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.452682 11581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:37.471515 11581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:37.472262 11581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:37.472471 11581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:37.479503 11581 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:46701
I20260808 15:52:37.479511 11674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:46701 every 8 connection(s)
I20260808 15:52:37.480581 11581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:37.483805 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:37.486720 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 11581
I20260808 15:52:37.486840 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:37.487161 31025 external_mini_cluster.cc:1428] Setting key 39f91237557166225934485ed54bf5ab
I20260808 15:52:37.489750 11675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9: Bootstrap starting.
I20260808 15:52:37.492004 11675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:37.492769 11675 log.cc:824] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:37.494692 11675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9: No bootstrap required, opened a new log
I20260808 15:52:37.496568 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:37.488569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.30.76.126:46701)
0808 15:52:37.489246 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:37.489261 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:37.489295 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:52:37.489963 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:37.489981 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:37.490356 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:37.490702 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:37.491134 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.492237 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.493190 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.493662 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.493860 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.494727 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:37.494732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:37.494744 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:37.495246 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:37.495336 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:37.495359 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:37.495467 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:37.495645 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 15:52:37.495893 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":138,"threads_started":1}
I20260808 15:52:37.498778 11675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46701 } }
I20260808 15:52:37.499050 11675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:37.499132 11675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3d27eacbbe488393fc1d7bbbebdbc9, State: Initialized, Role: FOLLOWER
I20260808 15:52:37.499516 11675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46701 } }
I20260808 15:52:37.499720 11675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:37.499809 11675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:37.499908 11675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:37.500751 11675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46701 } }
I20260808 15:52:37.501089 11675 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3d27eacbbe488393fc1d7bbbebdbc9; no voters:
I20260808 15:52:37.501384 11675 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:37.501492 11684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:37.501778 11684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [term 1 LEADER]: Becoming Leader. State: Replica: 6c3d27eacbbe488393fc1d7bbbebdbc9, State: Running, Role: LEADER
I20260808 15:52:37.502107 11684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46701 } }
I20260808 15:52:37.502564 11675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:37.503592 11685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46701 } } }
I20260808 15:52:37.503715 11685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:37.504189 11686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3d27eacbbe488393fc1d7bbbebdbc9. Latest consensus state: current_term: 1 leader_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3d27eacbbe488393fc1d7bbbebdbc9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46701 } } }
I20260808 15:52:37.504271 11697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:37.504361 11686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:37.507462 11697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:37.512862 11697 catalog_manager.cc:1347] Generated new cluster ID: 585a1d91de33480db9e149b01943c830
I20260808 15:52:37.512925 11697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:37.539515 11697 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:37.540858 11697 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:37.551321 11697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c3d27eacbbe488393fc1d7bbbebdbc9: Generated new TSK 0
I20260808 15:52:37.552162 11697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:52:37.554816 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46701
--builtin_ntp_servers=127.30.76.84:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 15:52:37.668337 11711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:37.668609 11711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:37.668678 11711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:37.672788 11711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:37.672866 11711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:37.672973 11711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:37.677570 11711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:37.678661 11711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:37.680083 11711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:37.689738 11718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:37.689746 11719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:37.689751 11722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:37.689821 11711 server_base.cc:1034] running on GCE node
I20260808 15:52:37.690459 11711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:37.691159 11711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:37.692349 11711 hybrid_clock.cc:648] HybridClock initialized: now 1786204357692310 us; error 61 us; skew 500 ppm
I20260808 15:52:37.694181 11711 webserver.cc:460] Webserver started at http://127.30.76.65:34623/ using document root <none> and password file <none>
I20260808 15:52:37.694756 11711 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:37.694847 11711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:37.695097 11711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:37.697172 11711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "a6b0bf5f3ac444e5bfcdd4e41585a307"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "2febb9c9bf06ad9489f121e16bb5f87f"
server_key_iv: "b9a07aa4ced4b3bd82fc072c26725aa2"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.697726 11711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "a6b0bf5f3ac444e5bfcdd4e41585a307"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "2febb9c9bf06ad9489f121e16bb5f87f"
server_key_iv: "b9a07aa4ced4b3bd82fc072c26725aa2"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.701328 11711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 15:52:37.703910 11732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.705193 11711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:37.705340 11711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "a6b0bf5f3ac444e5bfcdd4e41585a307"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "2febb9c9bf06ad9489f121e16bb5f87f"
server_key_iv: "b9a07aa4ced4b3bd82fc072c26725aa2"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.705454 11711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:37.737285 11711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:37.738075 11711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:37.738304 11711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:37.738932 11711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:37.739943 11711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:37.740017 11711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.740103 11711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:37.740166 11711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.750078 11711 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:36719
I20260808 15:52:37.750113 11885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:36719 every 8 connection(s)
I20260808 15:52:37.751019 11711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:37.752552 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 11711
I20260808 15:52:37.752648 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:37.752979 31025 external_mini_cluster.cc:1428] Setting key 05c193e3952c87bea3db0bcb419fd255
I20260808 15:52:37.755142 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46701
--builtin_ntp_servers=127.30.76.84:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 15:52:37.761335 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:37.753943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:56543 (local address 127.30.76.126:46701)
0808 15:52:37.754062 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:37.754066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:37.754395 (+ 329us) server_negotiation.cc:407] Connection header received
0808 15:52:37.756046 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:37.756052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:37.756118 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:37.756218 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:37.757324 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.757872 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.759257 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.759498 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.759675 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.760802 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:37.760806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:37.760809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:37.760822 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:37.760852 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:37.760861 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:37.760964 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:37.761126 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 15:52:37.761191 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:52:37.762156 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:37.753472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:46701 (local address 127.30.76.65:56543)
0808 15:52:37.754233 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:37.754287 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:37.755811 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:37.756374 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:37.756380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:37.756479 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:37.757150 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:37.757173 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.758012 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.758018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:37.759124 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:37.759135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.759714 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.759718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:37.759896 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.760537 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:37.760539 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:37.760579 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:37.760642 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:37.760959 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:37.760970 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:37.760979 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 15:52:37.761113 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 15:52:37.761385 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":146,"threads_started":1}
I20260808 15:52:37.763427 11886 heartbeater.cc:347] Connected to a master server at 127.30.76.126:46701
I20260808 15:52:37.763715 11886 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:37.764256 11886 heartbeater.cc:511] Master 127.30.76.126:46701 requested a full tablet report, sending...
I20260808 15:52:37.766012 11621 ts_manager.cc:194] Registered new tserver with Master: a6b0bf5f3ac444e5bfcdd4e41585a307 (127.30.76.65:36719)
I20260808 15:52:37.767817 11621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.65:56543
W20260808 15:52:37.868713 11889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:37.868939 11889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:37.868973 11889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:37.872920 11889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:37.872963 11889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:37.873010 11889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:37.877394 11889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:37.878285 11889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:37.879482 11889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:37.889046 11901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:37.889017 11896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:37.889139 11889 server_base.cc:1034] running on GCE node
W20260808 15:52:37.889017 11898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:37.889747 11889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:37.890450 11889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:37.891603 11889 hybrid_clock.cc:648] HybridClock initialized: now 1786204357891566 us; error 48 us; skew 500 ppm
I20260808 15:52:37.893309 11889 webserver.cc:460] Webserver started at http://127.30.76.66:44627/ using document root <none> and password file <none>
I20260808 15:52:37.893819 11889 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:37.893880 11889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:37.894037 11889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:37.896108 11889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "4fc6679466bd4ad9a70d0bbb089100ff"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "39393b6b6388328cfcaffaf0e9e79e0a"
server_key_iv: "ea1e025239f631c59800b746e82c78b9"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.896607 11889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "4fc6679466bd4ad9a70d0bbb089100ff"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "39393b6b6388328cfcaffaf0e9e79e0a"
server_key_iv: "ea1e025239f631c59800b746e82c78b9"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.900051 11889 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:52:37.902412 11909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.904047 11889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:52:37.904196 11889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "4fc6679466bd4ad9a70d0bbb089100ff"
format_stamp: "Formatted at 2026-08-08 15:52:37 on dist-test-slave-sb2z"
server_key: "39393b6b6388328cfcaffaf0e9e79e0a"
server_key_iv: "ea1e025239f631c59800b746e82c78b9"
server_key_version: "encryptionkey@0"
I20260808 15:52:37.904333 11889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:37.918254 11889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:37.918885 11889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:37.919077 11889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:37.919723 11889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:37.920639 11889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:37.920709 11889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.920781 11889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:37.920825 11889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:37.929292 11889 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:35873
I20260808 15:52:37.929317 12063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:35873 every 8 connection(s)
I20260808 15:52:37.930166 11889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:37.932722 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 11889
I20260808 15:52:37.932838 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:37.933157 31025 external_mini_cluster.cc:1428] Setting key 1313114149a218a6d685d0dac3cdb420
I20260808 15:52:37.935277 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46701
--builtin_ntp_servers=127.30.76.84:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 15:52:37.940943 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:37.931969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:55281 (local address 127.30.76.126:46701)
0808 15:52:37.932136 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:37.932142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:37.933290 (+ 1148us) server_negotiation.cc:407] Connection header received
0808 15:52:37.934903 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:37.934906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:37.934958 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:37.935051 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:37.936048 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.936887 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.938708 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.939088 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.939223 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.940354 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:37.940361 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:37.940365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:37.940383 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:37.940407 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:37.940418 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:37.940502 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:37.940648 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 15:52:37.940724 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:52:37.941661 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:37.932376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:46701 (local address 127.30.76.66:55281)
0808 15:52:37.933133 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:37.933173 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:37.934702 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:37.935195 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:37.935202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:37.935280 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:37.935897 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:37.935911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.937063 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.937069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:37.938544 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:37.938559 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:37.939249 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:37.939252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:37.939388 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:37.940068 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:37.940069 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:37.940112 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:37.940187 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:37.940502 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:37.940510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:37.940515 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:52:37.940624 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 15:52:37.940835 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":142,"threads_started":1}
I20260808 15:52:37.942806 12064 heartbeater.cc:347] Connected to a master server at 127.30.76.126:46701
I20260808 15:52:37.943037 12064 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:37.943588 12064 heartbeater.cc:511] Master 127.30.76.126:46701 requested a full tablet report, sending...
I20260808 15:52:37.944671 11621 ts_manager.cc:194] Registered new tserver with Master: 4fc6679466bd4ad9a70d0bbb089100ff (127.30.76.66:35873)
I20260808 15:52:37.945318 11621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.66:55281
W20260808 15:52:38.051496 12067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:38.051842 12067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:38.051949 12067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:38.056456 12067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:38.056546 12067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:38.056627 12067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:38.061470 12067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:38.062482 12067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:38.063902 12067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:38.073648 12075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.073654 12074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:38.073745 12067 server_base.cc:1034] running on GCE node
W20260808 15:52:38.073693 12077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:38.074309 12067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:38.074903 12067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:38.076078 12067 hybrid_clock.cc:648] HybridClock initialized: now 1786204358076062 us; error 27 us; skew 500 ppm
I20260808 15:52:38.077874 12067 webserver.cc:460] Webserver started at http://127.30.76.67:39041/ using document root <none> and password file <none>
I20260808 15:52:38.078394 12067 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:38.078454 12067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:38.078616 12067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:38.080595 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "cb20ee1d8c664287918ac0a823317629"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "5b10e808e3debe98679c94660b48a9b6"
server_key_iv: "28a132d55fed2552b67633a880f865ec"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.081123 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "cb20ee1d8c664287918ac0a823317629"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "5b10e808e3debe98679c94660b48a9b6"
server_key_iv: "28a132d55fed2552b67633a880f865ec"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.084499 12067 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:38.086894 12085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.088102 12067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:38.088193 12067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "cb20ee1d8c664287918ac0a823317629"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "5b10e808e3debe98679c94660b48a9b6"
server_key_iv: "28a132d55fed2552b67633a880f865ec"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.088274 12067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:38.118685 12067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:38.119365 12067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:38.119517 12067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:38.120138 12067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:38.121044 12067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:38.121093 12067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.121134 12067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:38.121150 12067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.130693 12067 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44665
I20260808 15:52:38.130769 12235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44665 every 8 connection(s)
I20260808 15:52:38.131630 12067 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:38.133140 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 12067
I20260808 15:52:38.133260 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:38.133599 31025 external_mini_cluster.cc:1428] Setting key 713ac222c9f494b24db6be4c2162839c
I20260808 15:52:38.140846 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.133531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:48803 (local address 127.30.76.126:46701)
0808 15:52:38.133676 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.133683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.134805 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 15:52:38.136266 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.136269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.136326 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.136433 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.137281 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:38.137805 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:38.138965 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:38.139189 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:38.139296 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:38.140354 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:38.140357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:38.140360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:38.140379 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:38.140397 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:38.140404 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:38.140485 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:38.140643 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 15:52:38.140704 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:52:38.141642 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.133944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:46701 (local address 127.30.76.67:48803)
0808 15:52:38.134663 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:38.134714 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:38.136069 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:38.136606 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:38.136613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:38.136691 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:38.137142 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:38.137154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:38.137931 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:38.137934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:38.138831 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:38.138841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:38.139346 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:38.139351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:38.139514 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:38.140107 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:38.140108 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:38.140145 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:38.140205 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:38.140487 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:38.140499 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:38.140506 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:52:38.140641 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:52:38.140904 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":141,"threads_started":1}
I20260808 15:52:38.142688 12236 heartbeater.cc:347] Connected to a master server at 127.30.76.126:46701
I20260808 15:52:38.142894 12236 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:38.143327 12236 heartbeater.cc:511] Master 127.30.76.126:46701 requested a full tablet report, sending...
I20260808 15:52:38.144330 11621 ts_manager.cc:194] Registered new tserver with Master: cb20ee1d8c664287918ac0a823317629 (127.30.76.67:44665)
I20260808 15:52:38.144867 11621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.67:48803
I20260808 15:52:38.147294 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 15:52:38.149969 11679 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:38.149175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.30.76.126:46701)
0808 15:52:38.149321 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.149326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.149444 (+ 118us) server_negotiation.cc:407] Connection header received
0808 15:52:38.149545 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.149547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.149583 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.149643 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.149855 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43010: BlockingRecv error: recv got EOF from 127.0.0.1:43010 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:52:38.150350 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 11711
I20260808 15:52:38.164918 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 11889
I20260808 15:52:38.175366 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 12067
I20260808 15:52:38.185312 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 11581
2026-08-08T15:52:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (920 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T15:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:38Z Disabled control of system clock
I20260808 15:52:38.221814 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:34305
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 15:52:38.332317 12252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:38.332551 12252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:38.332587 12252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:38.336405 12252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:38.336452 12252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:38.336467 12252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:38.336483 12252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:38.336498 12252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:38.341102 12252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:34305
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:38.342013 12252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:38.343276 12252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:38.351359 12263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.351367 12261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.351397 12265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:38.351747 12252 server_base.cc:1034] running on GCE node
I20260808 15:52:38.352394 12252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:38.353359 12252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:38.354528 12252 hybrid_clock.cc:648] HybridClock initialized: now 1786204358354500 us; error 39 us; skew 500 ppm
I20260808 15:52:38.356287 12252 webserver.cc:460] Webserver started at http://127.30.76.126:46835/ using document root <none> and password file <none>
I20260808 15:52:38.356791 12252 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:38.356848 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:38.357069 12252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:38.359009 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "cfbcdbca34af4249a3d170141c9be51c"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "77435fa7160f5c423b1b73b82d819827"
server_key_iv: "0818c5463a3ee262e741f01b4870e846"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.359623 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "cfbcdbca34af4249a3d170141c9be51c"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "77435fa7160f5c423b1b73b82d819827"
server_key_iv: "0818c5463a3ee262e741f01b4870e846"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.363166 12252 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:52:38.365483 12271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.366575 12252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:38.366704 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "cfbcdbca34af4249a3d170141c9be51c"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "77435fa7160f5c423b1b73b82d819827"
server_key_iv: "0818c5463a3ee262e741f01b4870e846"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.366802 12252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:38.371207 12252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:38.371830 12252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:38.372038 12252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:38.378635 12252 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:34305
I20260808 15:52:38.378643 12343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:34305 every 8 connection(s)
I20260808 15:52:38.379689 12252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:38.382874 12345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:38.387590 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 12252
I20260808 15:52:38.387717 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:38.388015 31025 external_mini_cluster.cc:1428] Setting key 5d69758d3c2576681131599207abb20d
I20260808 15:52:38.388562 12345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c: Bootstrap starting.
I20260808 15:52:38.390990 12345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:38.391984 12345 log.cc:824] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:38.393471 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.389402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.30.76.126:34305)
0808 15:52:38.390005 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.390021 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.390056 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:52:38.390660 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.390675 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.390997 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.391324 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.392187 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:38.392191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:38.392202 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:38.392214 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:38.392267 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:38.392282 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:38.392369 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:38.392563 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 15:52:38.392822 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":133,"threads_started":1}
I20260808 15:52:38.393970 12345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c: No bootstrap required, opened a new log
I20260808 15:52:38.397768 12345 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbcdbca34af4249a3d170141c9be51c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34305 } }
I20260808 15:52:38.397992 12345 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:38.398074 12345 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbcdbca34af4249a3d170141c9be51c, State: Initialized, Role: FOLLOWER
I20260808 15:52:38.398468 12345 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbcdbca34af4249a3d170141c9be51c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34305 } }
I20260808 15:52:38.398613 12345 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:38.398705 12345 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:38.398803 12345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:38.399698 12345 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbcdbca34af4249a3d170141c9be51c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34305 } }
I20260808 15:52:38.400015 12345 leader_election.cc:302] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfbcdbca34af4249a3d170141c9be51c; no voters:
I20260808 15:52:38.400346 12345 leader_election.cc:288] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:38.400463 12351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:38.400671 12351 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [term 1 LEADER]: Becoming Leader. State: Replica: cfbcdbca34af4249a3d170141c9be51c, State: Running, Role: LEADER
I20260808 15:52:38.400955 12351 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbcdbca34af4249a3d170141c9be51c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34305 } }
I20260808 15:52:38.401542 12345 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:38.402583 12352 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfbcdbca34af4249a3d170141c9be51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbcdbca34af4249a3d170141c9be51c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34305 } } }
I20260808 15:52:38.402607 12353 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfbcdbca34af4249a3d170141c9be51c. Latest consensus state: current_term: 1 leader_uuid: "cfbcdbca34af4249a3d170141c9be51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbcdbca34af4249a3d170141c9be51c" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 34305 } } }
I20260808 15:52:38.402765 12353 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:38.402765 12352 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:38.403191 12361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:38.406220 12361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:38.411664 12361 catalog_manager.cc:1347] Generated new cluster ID: a71600dbdb394028a4932b785da1de0e
I20260808 15:52:38.411726 12361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:38.429709 12361 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:38.430012 12361 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:38.443979 12361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfbcdbca34af4249a3d170141c9be51c: Generated new TSK 0
I20260808 15:52:38.444832 12361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:52:38.457871 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34305
--builtin_ntp_servers=127.30.76.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 15:52:38.570885 12373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:38.571201 12373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:38.571295 12373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:38.575474 12373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:38.575574 12373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:38.575664 12373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:38.580329 12373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:38.581395 12373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:38.582778 12373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:38.592360 12381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.592401 12382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.592367 12384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:38.592711 12373 server_base.cc:1034] running on GCE node
I20260808 15:52:38.593097 12373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:38.593714 12373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:38.594903 12373 hybrid_clock.cc:648] HybridClock initialized: now 1786204358594882 us; error 33 us; skew 500 ppm
I20260808 15:52:38.596684 12373 webserver.cc:460] Webserver started at http://127.30.76.65:37031/ using document root <none> and password file <none>
I20260808 15:52:38.597295 12373 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:38.597393 12373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:38.597627 12373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:38.599679 12373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "8e68f21a8ed94efea395acd7cc17207d"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "f505bf3e91a046d635500d4773e835c7"
server_key_iv: "e0026ebcd449bb45385a6f49acbfeaac"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.600239 12373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "8e68f21a8ed94efea395acd7cc17207d"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "f505bf3e91a046d635500d4773e835c7"
server_key_iv: "e0026ebcd449bb45385a6f49acbfeaac"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.603932 12373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:52:38.606370 12393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.607515 12373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:38.607720 12373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "8e68f21a8ed94efea395acd7cc17207d"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "f505bf3e91a046d635500d4773e835c7"
server_key_iv: "e0026ebcd449bb45385a6f49acbfeaac"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.607833 12373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:38.612290 12373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:38.612890 12373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:38.613076 12373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:38.613612 12373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:38.614500 12373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:38.614573 12373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.614646 12373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:38.614691 12373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.623054 12373 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:37713
I20260808 15:52:38.623080 12550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:37713 every 8 connection(s)
I20260808 15:52:38.624121 12373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:38.624924 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 12373
I20260808 15:52:38.625041 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:38.625367 31025 external_mini_cluster.cc:1428] Setting key df2f9514bb8a6cfc1f7a276d59c21fed
I20260808 15:52:38.627686 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34305
--builtin_ntp_servers=127.30.76.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 15:52:38.630482 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.626040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:37689 (local address 127.30.76.126:34305)
0808 15:52:38.626187 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.626192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.627299 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 15:52:38.628732 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.628735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.628776 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.628881 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.630053 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:38.630056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:38.630058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:38.630070 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:38.630092 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:38.630100 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:38.630180 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:38.630344 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 15:52:38.630381 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:52:38.631593 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.626412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:34305 (local address 127.30.76.65:37689)
0808 15:52:38.627158 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:38.627197 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:38.628518 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:38.629038 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:38.629044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:38.629147 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:38.629744 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:38.629748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:38.629801 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:38.629888 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:38.630188 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:38.630198 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:38.630207 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 15:52:38.630382 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 15:52:38.630633 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260808 15:52:38.632742 12551 heartbeater.cc:347] Connected to a master server at 127.30.76.126:34305
I20260808 15:52:38.632989 12551 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:38.633476 12551 heartbeater.cc:511] Master 127.30.76.126:34305 requested a full tablet report, sending...
I20260808 15:52:38.635027 12295 ts_manager.cc:194] Registered new tserver with Master: 8e68f21a8ed94efea395acd7cc17207d (127.30.76.65:37713)
W20260808 15:52:38.740849 12556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:38.741149 12556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:38.741230 12556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:38.745296 12556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:38.745379 12556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:38.745466 12556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:38.750233 12556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:38.751329 12556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:38.752732 12556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:38.762422 12566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.762422 12565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.762493 12569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:38.762645 12556 server_base.cc:1034] running on GCE node
I20260808 15:52:38.763247 12556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:38.763947 12556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:38.765143 12556 hybrid_clock.cc:648] HybridClock initialized: now 1786204358765131 us; error 35 us; skew 500 ppm
I20260808 15:52:38.771591 12556 webserver.cc:460] Webserver started at http://127.30.76.66:46655/ using document root <none> and password file <none>
I20260808 15:52:38.772192 12556 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:38.772257 12556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:38.772559 12556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:38.775173 12556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "a41890475c9d49b1baffc90ad00e4c42"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "6c8527fce2b9583107ae6bca315b9142"
server_key_iv: "e70f4b2426dfa2277b0c95ec5fe69f6a"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.775803 12556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "a41890475c9d49b1baffc90ad00e4c42"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "6c8527fce2b9583107ae6bca315b9142"
server_key_iv: "e70f4b2426dfa2277b0c95ec5fe69f6a"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.779451 12556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:52:38.781922 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.783082 12556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:38.783213 12556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "a41890475c9d49b1baffc90ad00e4c42"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "6c8527fce2b9583107ae6bca315b9142"
server_key_iv: "e70f4b2426dfa2277b0c95ec5fe69f6a"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.783313 12556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:38.787853 12556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:38.788451 12556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:38.788628 12556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:38.789165 12556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:38.790038 12556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:38.790086 12556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.790170 12556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:38.790208 12556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.798741 12556 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:33751
I20260808 15:52:38.798779 12739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:33751 every 8 connection(s)
I20260808 15:52:38.799628 12556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:38.805261 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 12556
I20260808 15:52:38.805408 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:38.805624 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.801330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:36299 (local address 127.30.76.126:34305)
0808 15:52:38.801480 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.801484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.802583 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 15:52:38.804004 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.804007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.804046 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.804124 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.805103 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:38.805107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:38.805109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:38.805125 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:38.805163 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:38.805174 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:38.805279 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:38.805429 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 15:52:38.805479 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:52:38.805763 31025 external_mini_cluster.cc:1428] Setting key 46af0dd6c893721b2d8441e01b71bb68
I20260808 15:52:38.806428 12743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.801700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:34305 (local address 127.30.76.66:36299)
0808 15:52:38.802427 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:38.802484 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:38.803817 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:38.804274 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:38.804280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:38.804361 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:38.804849 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:38.804850 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:38.804887 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:38.804950 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:38.805275 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:38.805280 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:38.805283 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 15:52:38.805453 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 15:52:38.805661 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260808 15:52:38.807471 12741 heartbeater.cc:347] Connected to a master server at 127.30.76.126:34305
I20260808 15:52:38.807713 12741 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:38.807958 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34305
--builtin_ntp_servers=127.30.76.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 15:52:38.808161 12741 heartbeater.cc:511] Master 127.30.76.126:34305 requested a full tablet report, sending...
I20260808 15:52:38.809207 12295 ts_manager.cc:194] Registered new tserver with Master: a41890475c9d49b1baffc90ad00e4c42 (127.30.76.66:33751)
W20260808 15:52:38.920327 12744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:38.920567 12744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:38.920639 12744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:38.924606 12744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:38.924659 12744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:38.924763 12744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:38.929402 12744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:38.930408 12744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:38.931782 12744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:38.940941 12751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.940973 12755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:38.941026 12753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:38.941252 12744 server_base.cc:1034] running on GCE node
I20260808 15:52:38.941651 12744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:38.942270 12744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:38.943452 12744 hybrid_clock.cc:648] HybridClock initialized: now 1786204358943436 us; error 38 us; skew 500 ppm
I20260808 15:52:38.945237 12744 webserver.cc:460] Webserver started at http://127.30.76.67:37757/ using document root <none> and password file <none>
I20260808 15:52:38.945808 12744 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:38.945899 12744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:38.946118 12744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:38.948261 12744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "18fa1a23ad9c4f5fb96552b6ff4db872"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "463fbf6a636ebd2be8f1a4e1c013114d"
server_key_iv: "c028383a7cc73bc5f7e20d3245de306a"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.948881 12744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "18fa1a23ad9c4f5fb96552b6ff4db872"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "463fbf6a636ebd2be8f1a4e1c013114d"
server_key_iv: "c028383a7cc73bc5f7e20d3245de306a"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.952474 12744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:38.954846 12762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.956092 12744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:52:38.956221 12744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "18fa1a23ad9c4f5fb96552b6ff4db872"
format_stamp: "Formatted at 2026-08-08 15:52:38 on dist-test-slave-sb2z"
server_key: "463fbf6a636ebd2be8f1a4e1c013114d"
server_key_iv: "c028383a7cc73bc5f7e20d3245de306a"
server_key_version: "encryptionkey@0"
I20260808 15:52:38.956331 12744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:38.961002 12744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:38.961565 12744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:38.961743 12744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:38.962255 12744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:38.963104 12744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:38.963172 12744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.963238 12744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:38.963287 12744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:38.972446 12744 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:39429
I20260808 15:52:38.972505 12912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:39429 every 8 connection(s)
I20260808 15:52:38.973503 12744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:38.975303 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 12744
I20260808 15:52:38.975430 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:38.975795 31025 external_mini_cluster.cc:1428] Setting key 6c15954049449701c2db8ecbea393b67
I20260808 15:52:38.980067 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.975356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:45357 (local address 127.30.76.126:34305)
0808 15:52:38.977015 (+ 1659us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.977020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.977038 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:52:38.978363 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.978367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.978413 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.978507 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.979547 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:38.979554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:38.979560 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:52:38.979574 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:38.979598 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:52:38.979609 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:38.979725 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:38.979882 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 15:52:38.979930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1561}
I20260808 15:52:38.981196 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:38.975960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:34305 (local address 127.30.76.67:45357)
0808 15:52:38.976782 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:38.976837 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:38.978142 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:38.978664 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:38.978671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:38.978761 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:38.979255 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:52:38.979257 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:38.979295 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:52:38.979362 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:38.979713 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:38.979723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:38.979730 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:52:38.979925 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 15:52:38.980186 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":166,"threads_started":1}
I20260808 15:52:38.982328 12913 heartbeater.cc:347] Connected to a master server at 127.30.76.126:34305
I20260808 15:52:38.982558 12913 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:38.982952 12913 heartbeater.cc:511] Master 127.30.76.126:34305 requested a full tablet report, sending...
I20260808 15:52:38.983971 12295 ts_manager.cc:194] Registered new tserver with Master: 18fa1a23ad9c4f5fb96552b6ff4db872 (127.30.76.67:39429)
I20260808 15:52:38.989722 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 15:52:38.992575 12349 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:52:38.991716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.30.76.126:34305)
0808 15:52:38.991840 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:38.991843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:38.992041 (+ 198us) server_negotiation.cc:407] Connection header received
0808 15:52:38.992146 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:38.992148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:38.992176 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:38.992241 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:38.992435 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37442: BlockingRecv error: recv got EOF from 127.0.0.1:37442 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:38.992816 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 12373
I20260808 15:52:39.006460 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 12556
I20260808 15:52:39.014837 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 12744
I20260808 15:52:39.025995 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 12252
2026-08-08T15:52:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (842 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:39 dist-test-slave-sb2z krb5kdc[12929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:39 dist-test-slave-sb2z krb5kdc[12929](info): set up 2 sockets
Aug 08 15:52:39 dist-test-slave-sb2z krb5kdc[12929](info): commencing operation
krb5kdc: starting...
W20260808 15:52:40.267100 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:40 dist-test-slave-sb2z krb5kdc[12929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:52:40.390407 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36543
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:36543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:40.499704 12947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:40.499954 12947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:40.500031 12947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:40.503950 12947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:52:40.503997 12947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:40.504020 12947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:40.504081 12947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:52:40.504106 12947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:52:40.509222 12947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:36543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:40.510196 12947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:40.511204 12947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:52:40.519680 12947 server_base.cc:1034] running on GCE node
W20260808 15:52:40.519654 12957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:40.519644 12959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:40.519644 12956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:40.520571 12947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:40.521729 12947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:40.522894 12947 hybrid_clock.cc:648] HybridClock initialized: now 1786204360522875 us; error 34 us; skew 500 ppm
Aug 08 15:52:40 dist-test-slave-sb2z krb5kdc[12929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204360, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:40.530043 12947 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:52:40.530970 12947 webserver.cc:460] Webserver started at http://127.30.76.126:46329/ using document root <none> and password file <none>
I20260808 15:52:40.531502 12947 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:40.531621 12947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:40.531818 12947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:40.533867 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "aa4f02bb61b34572a8db8993fce027a7"
format_stamp: "Formatted at 2026-08-08 15:52:40 on dist-test-slave-sb2z"
server_key: "2d1573494fc361395102262dd37a4f58"
server_key_iv: "92e974212273e0c8fd4952f3735f72b5"
server_key_version: "encryptionkey@0"
I20260808 15:52:40.534415 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "aa4f02bb61b34572a8db8993fce027a7"
format_stamp: "Formatted at 2026-08-08 15:52:40 on dist-test-slave-sb2z"
server_key: "2d1573494fc361395102262dd37a4f58"
server_key_iv: "92e974212273e0c8fd4952f3735f72b5"
server_key_version: "encryptionkey@0"
I20260808 15:52:40.538194 12947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:52:40.540618 12972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:40.541991 12947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:40.542145 12947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "aa4f02bb61b34572a8db8993fce027a7"
format_stamp: "Formatted at 2026-08-08 15:52:40 on dist-test-slave-sb2z"
server_key: "2d1573494fc361395102262dd37a4f58"
server_key_iv: "92e974212273e0c8fd4952f3735f72b5"
server_key_version: "encryptionkey@0"
I20260808 15:52:40.542263 12947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:40.573918 12947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:40.577343 12947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:40.577557 12947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:40.584649 12947 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:36543
I20260808 15:52:40.584942 13035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:36543 every 8 connection(s)
I20260808 15:52:40.585645 12947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:52:40.586786 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 12947
I20260808 15:52:40.586925 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:52:40.587244 31025 external_mini_cluster.cc:1428] Setting key 073f596365e94b137b280c07f9506572
I20260808 15:52:40.588999 13036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:40.595973 13036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7: Bootstrap starting.
Aug 08 15:52:40 dist-test-slave-sb2z krb5kdc[12929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:40.598771 13036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:40.599745 13036 log.cc:824] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:40.601845 13036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7: No bootstrap required, opened a new log
I20260808 15:52:40.602425 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:40.588645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.30.76.126:36543)
0808 15:52:40.589083 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:40.589103 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:40.589147 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:52:40.589995 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:40.590020 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:40.590458 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:40.590839 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:40.591566 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:40.593084 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:40.593935 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:40.594624 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:40.594803 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:40.597340 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:40.597347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:40.597364 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:40.597396 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:40.599438 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:40.599986 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:40.599991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:40.599996 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:40.600114 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:40.600410 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:40.600414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:40.600416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:40.600832 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:40.601046 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:40.601361 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 15:52:40.601653 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":115,"threads_started":1}
I20260808 15:52:40.605870 13036 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa4f02bb61b34572a8db8993fce027a7" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36543 } }
I20260808 15:52:40.606144 13036 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:40.606211 13036 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4f02bb61b34572a8db8993fce027a7, State: Initialized, Role: FOLLOWER
I20260808 15:52:40.606753 13036 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa4f02bb61b34572a8db8993fce027a7" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36543 } }
I20260808 15:52:40.606926 13036 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:52:40.606983 13036 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:52:40.607061 13036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:40.608018 13036 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa4f02bb61b34572a8db8993fce027a7" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36543 } }
I20260808 15:52:40.608312 13036 leader_election.cc:302] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa4f02bb61b34572a8db8993fce027a7; no voters:
I20260808 15:52:40.608577 13036 leader_election.cc:288] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:52:40.608733 13042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:40.608978 13042 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [term 1 LEADER]: Becoming Leader. State: Replica: aa4f02bb61b34572a8db8993fce027a7, State: Running, Role: LEADER
I20260808 15:52:40.609349 13042 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa4f02bb61b34572a8db8993fce027a7" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36543 } }
I20260808 15:52:40.609910 13036 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:52:40.611166 13044 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa4f02bb61b34572a8db8993fce027a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa4f02bb61b34572a8db8993fce027a7" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36543 } } }
I20260808 15:52:40.611163 13046 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa4f02bb61b34572a8db8993fce027a7. Latest consensus state: current_term: 1 leader_uuid: "aa4f02bb61b34572a8db8993fce027a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa4f02bb61b34572a8db8993fce027a7" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36543 } } }
I20260808 15:52:40.611393 13046 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:40.611393 13044 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7 [sys.catalog]: This master's current role is: LEADER
I20260808 15:52:40.611914 13053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:52:40.614603 13053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:52:40.620368 13053 catalog_manager.cc:1347] Generated new cluster ID: 77b10d782f7c45a894fcfc8c62c09ef4
I20260808 15:52:40.620452 13053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:52:40.631948 13053 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:52:40.632834 13053 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:52:40.637578 13053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7: Generated new TSK 0
I20260808 15:52:40.638207 13053 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:52:40.672811 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36543
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:40.786682 13069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:40.786947 13069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:40.787042 13069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:40.791010 13069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:40.791092 13069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:40.791181 13069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:52:40.796283 13069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:40.797274 13069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:40.798285 13069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:40.807942 13077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:40.808054 13080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:40.807951 13078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:40.808257 13069 server_base.cc:1034] running on GCE node
I20260808 15:52:40.808715 13069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:40.809302 13069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:40.810524 13069 hybrid_clock.cc:648] HybridClock initialized: now 1786204360810508 us; error 29 us; skew 500 ppm
Aug 08 15:52:40 dist-test-slave-sb2z krb5kdc[12929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204360, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:40.813342 13069 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:52:40.814113 13069 webserver.cc:460] Webserver started at http://127.30.76.65:34391/ using document root <none> and password file <none>
I20260808 15:52:40.814617 13069 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:40.814677 13069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:40.814834 13069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:40.816807 13069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a"
format_stamp: "Formatted at 2026-08-08 15:52:40 on dist-test-slave-sb2z"
server_key: "3c0c1c31334e98ab1499a567eec46225"
server_key_iv: "1ccea6a8d4fd1ff1becfab13cbac1f4b"
server_key_version: "encryptionkey@0"
I20260808 15:52:40.817317 13069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a"
format_stamp: "Formatted at 2026-08-08 15:52:40 on dist-test-slave-sb2z"
server_key: "3c0c1c31334e98ab1499a567eec46225"
server_key_iv: "1ccea6a8d4fd1ff1becfab13cbac1f4b"
server_key_version: "encryptionkey@0"
I20260808 15:52:40.820858 13069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:52:40.823194 13087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:40.824414 13069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:52:40.824580 13069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a"
format_stamp: "Formatted at 2026-08-08 15:52:40 on dist-test-slave-sb2z"
server_key: "3c0c1c31334e98ab1499a567eec46225"
server_key_iv: "1ccea6a8d4fd1ff1becfab13cbac1f4b"
server_key_version: "encryptionkey@0"
I20260808 15:52:40.824674 13069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:40.847352 13069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:40.850678 13069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:40.850883 13069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:40.851473 13069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:40.852358 13069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:40.852407 13069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:40.852475 13069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:40.852515 13069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:40.861308 13069 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:42383
I20260808 15:52:40.861331 13248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:42383 every 8 connection(s)
I20260808 15:52:40.862231 13069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:52:40.869952 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 13069
I20260808 15:52:40.870079 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:52:40.870375 31025 external_mini_cluster.cc:1428] Setting key 1626361b1964b2813eb38f4dc4ee480f
Aug 08 15:52:40 dist-test-slave-sb2z krb5kdc[12929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204360, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:40.876361 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:40.864106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:34151 (local address 127.30.76.126:36543)
0808 15:52:40.864252 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:40.864256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:40.865058 (+ 802us) server_negotiation.cc:407] Connection header received
0808 15:52:40.866483 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:40.866486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:40.866544 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:40.866663 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:40.867778 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:40.868336 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:40.869465 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:40.869782 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:40.869901 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:40.872719 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:40.872722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:40.872724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:40.872746 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:40.874107 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:40.874607 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:40.874612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:40.874613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:40.874702 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:40.875045 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:40.875050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:40.875051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:40.875321 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:40.875423 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:40.875981 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 15:52:40.876125 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:52:40.877003 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:40.864465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36543 (local address 127.30.76.65:34151)
0808 15:52:40.864908 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:40.864950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:40.866258 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:40.866826 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:40.866833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:40.867176 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:40.867634 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:40.867647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:40.868468 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:40.868473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:40.869337 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:40.869346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:40.870053 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:40.870060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:40.870241 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:40.870863 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:40.870908 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:40.872544 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:40.874231 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:40.874236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:40.874250 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:40.874502 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:40.874797 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:40.874801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:40.874803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:40.874945 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:40.875447 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:40.875453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:40.875741 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 15:52:40.875898 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 15:52:40.876127 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260808 15:52:40.878353 13249 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36543
I20260808 15:52:40.878590 13249 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:40.879053 13249 heartbeater.cc:511] Master 127.30.76.126:36543 requested a full tablet report, sending...
I20260808 15:52:40.880738 12995 ts_manager.cc:194] Registered new tserver with Master: 50f2cd8c9f9d49e3b01cf6a854ef960a (127.30.76.65:42383)
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:52:40.882330 12995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:34151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:52:40.898777 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36543
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:41.008348 13255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:41.008594 13255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:41.008635 13255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:41.012742 13255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:41.012799 13255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:41.012853 13255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:52:41.018334 13255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:41.019677 13255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:41.021150 13255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:41.033201 13265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:41.033227 13262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:41.033319 13255 server_base.cc:1034] running on GCE node
W20260808 15:52:41.033211 13263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:41.033814 13255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:41.034432 13255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:41.035611 13255 hybrid_clock.cc:648] HybridClock initialized: now 1786204361035598 us; error 32 us; skew 500 ppm
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[12929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204361, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:41.038627 13255 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:52:41.039482 13255 webserver.cc:460] Webserver started at http://127.30.76.66:37157/ using document root <none> and password file <none>
I20260808 15:52:41.040042 13255 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:41.040138 13255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:41.040361 13255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:41.042290 13255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "175a90b7c7174c06818eb034e179bbcb"
format_stamp: "Formatted at 2026-08-08 15:52:41 on dist-test-slave-sb2z"
server_key: "c0424a275dae9d8d14866350e6ed9208"
server_key_iv: "3dd8377fe3e6b8617b9c8e3c4dde6c2a"
server_key_version: "encryptionkey@0"
I20260808 15:52:41.042816 13255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "175a90b7c7174c06818eb034e179bbcb"
format_stamp: "Formatted at 2026-08-08 15:52:41 on dist-test-slave-sb2z"
server_key: "c0424a275dae9d8d14866350e6ed9208"
server_key_iv: "3dd8377fe3e6b8617b9c8e3c4dde6c2a"
server_key_version: "encryptionkey@0"
I20260808 15:52:41.046478 13255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 15:52:41.048918 13273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:41.050060 13255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:52:41.050180 13255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "175a90b7c7174c06818eb034e179bbcb"
format_stamp: "Formatted at 2026-08-08 15:52:41 on dist-test-slave-sb2z"
server_key: "c0424a275dae9d8d14866350e6ed9208"
server_key_iv: "3dd8377fe3e6b8617b9c8e3c4dde6c2a"
server_key_version: "encryptionkey@0"
I20260808 15:52:41.050299 13255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:41.076049 13255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:41.079476 13255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:41.079720 13255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:41.080330 13255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:41.081171 13255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:41.081218 13255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:41.081288 13255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:41.081331 13255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:41.089721 13255 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:40161
I20260808 15:52:41.089744 13424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:40161 every 8 connection(s)
I20260808 15:52:41.090576 13255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:52:41.095073 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 13255
I20260808 15:52:41.095193 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:52:41.095501 31025 external_mini_cluster.cc:1428] Setting key ea68600d7784b7a73eac497accc7b822
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[12929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204361, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:41.105235 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.092395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:38057 (local address 127.30.76.126:36543)
0808 15:52:41.092543 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.092549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.093438 (+ 889us) server_negotiation.cc:407] Connection header received
0808 15:52:41.094780 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.094783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.094864 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.094968 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:41.096444 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.097093 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.098250 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.098478 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.098590 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.101265 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:41.101268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:41.101270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:41.101291 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:41.102897 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:41.103614 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:41.103617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:41.103618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:41.103692 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:41.104105 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:41.104107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:41.104109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:41.104284 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:41.104358 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.105011 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.105103 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":53}
I20260808 15:52:41.106397 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.092812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36543 (local address 127.30.76.66:38057)
0808 15:52:41.093292 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.093335 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.094582 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.095244 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.095256 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.095709 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:41.096286 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.096303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.097243 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.097246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.098109 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.098119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.098639 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.098644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.098814 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.099411 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:41.099427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:41.101083 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:41.103051 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:41.103060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:41.103080 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:41.103466 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:41.103789 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:41.103794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:41.103797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:41.103996 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:41.104396 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:41.104405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:41.104744 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.104964 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.105310 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:52:41.107692 13425 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36543
I20260808 15:52:41.107904 13425 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:41.108465 13425 heartbeater.cc:511] Master 127.30.76.126:36543 requested a full tablet report, sending...
I20260808 15:52:41.109442 12995 ts_manager.cc:194] Registered new tserver with Master: 175a90b7c7174c06818eb034e179bbcb (127.30.76.66:40161)
I20260808 15:52:41.110005 12995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:38057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:52:41.124571 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36543
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:52:41.236507 13431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:52:41.236791 13431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:52:41.236864 13431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:52:41.240846 13431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:52:41.240943 13431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:52:41.241060 13431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:52:41.246187 13431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:52:41.247233 13431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:52:41.248239 13431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:52:41.257385 13438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:41.257495 13431 server_base.cc:1034] running on GCE node
W20260808 15:52:41.257417 13440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:52:41.257413 13437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:52:41.257987 13431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:52:41.258589 13431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:52:41.259752 13431 hybrid_clock.cc:648] HybridClock initialized: now 1786204361259737 us; error 26 us; skew 500 ppm
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[12929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204361, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:52:41.262439 13431 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:52:41.263216 13431 webserver.cc:460] Webserver started at http://127.30.76.67:37459/ using document root <none> and password file <none>
I20260808 15:52:41.263783 13431 fs_manager.cc:359] Metadata directory not provided
I20260808 15:52:41.263846 13431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:52:41.264002 13431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:52:41.265883 13431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "3d5c7746b24f4031a45db1e2e84844f0"
format_stamp: "Formatted at 2026-08-08 15:52:41 on dist-test-slave-sb2z"
server_key: "2e4eb1b3bd6b4026252e6a32a3588208"
server_key_iv: "061a4ea5cedf148b4a01028e02e2fc5f"
server_key_version: "encryptionkey@0"
I20260808 15:52:41.266384 13431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "3d5c7746b24f4031a45db1e2e84844f0"
format_stamp: "Formatted at 2026-08-08 15:52:41 on dist-test-slave-sb2z"
server_key: "2e4eb1b3bd6b4026252e6a32a3588208"
server_key_iv: "061a4ea5cedf148b4a01028e02e2fc5f"
server_key_version: "encryptionkey@0"
I20260808 15:52:41.269979 13431 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:52:41.272327 13447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:41.273471 13431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:52:41.273592 13431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "3d5c7746b24f4031a45db1e2e84844f0"
format_stamp: "Formatted at 2026-08-08 15:52:41 on dist-test-slave-sb2z"
server_key: "2e4eb1b3bd6b4026252e6a32a3588208"
server_key_iv: "061a4ea5cedf148b4a01028e02e2fc5f"
server_key_version: "encryptionkey@0"
I20260808 15:52:41.273699 13431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:52:41.312726 13431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:52:41.315755 13431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:52:41.315913 13431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:52:41.316444 13431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:52:41.317682 13431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:52:41.317759 13431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:41.317838 13431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:52:41.317876 13431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:52:41.328291 13431 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:44771
I20260808 15:52:41.328343 13605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:44771 every 8 connection(s)
I20260808 15:52:41.329218 13431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:52:41.331655 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 13431
I20260808 15:52:41.331825 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:52:41.332173 31025 external_mini_cluster.cc:1428] Setting key 04649b9997416a0c0f0440188972a822
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[12929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204361, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:52:41.342736 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.331056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:44755 (local address 127.30.76.126:36543)
0808 15:52:41.331216 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.331221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.332314 (+ 1093us) server_negotiation.cc:407] Connection header received
0808 15:52:41.334089 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.334094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.334168 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.334275 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:41.335445 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.336017 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.337068 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.337279 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.337352 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.339784 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:41.339788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:41.339791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:41.339818 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:41.341041 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:41.341494 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:41.341498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:41.341500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:41.341586 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:41.341870 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:41.341872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:41.341873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:41.342035 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:41.342109 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.342528 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.342621 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:52:41.343505 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.331560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36543 (local address 127.30.76.67:44755)
0808 15:52:41.332146 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.332205 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.333887 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.334482 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.334497 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.334848 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:52:41.335321 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.335334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.336122 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.336131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.336967 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.336976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.337402 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.337405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.337540 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.338149 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:52:41.338165 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:52:41.339607 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:52:41.341145 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:41.341149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:41.341160 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:41.341399 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:41.341668 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:52:41.341670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:52:41.341671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:52:41.341793 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:52:41.342119 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:52:41.342131 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:52:41.342336 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.342482 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.342711 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":139,"threads_started":1}
I20260808 15:52:41.344833 13607 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36543
I20260808 15:52:41.345068 13607 heartbeater.cc:464] Registering TS with master...
I20260808 15:52:41.345703 13607 heartbeater.cc:511] Master 127.30.76.126:36543 requested a full tablet report, sending...
I20260808 15:52:41.346748 12995 ts_manager.cc:194] Registered new tserver with Master: 3d5c7746b24f4031a45db1e2e84844f0 (127.30.76.67:44771)
I20260808 15:52:41.347385 12995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:44755
I20260808 15:52:41.357785 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:52:41.366634 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.359767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.30.76.126:36543)
0808 15:52:41.359884 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.359888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.360134 (+ 246us) server_negotiation.cc:407] Connection header received
0808 15:52:41.360220 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.360222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.360264 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.360332 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:52:41.360985 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.361497 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.362299 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.362510 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.362599 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.364003 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:52:41.364006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:52:41.364008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:52:41.364031 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:52:41.365279 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:41.365642 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:41.365645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:41.365646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:41.365717 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:52:41.365938 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:52:41.365940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:52:41.365941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:52:41.366147 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:52:41.366247 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.366431 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.366522 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:52:41.369547 12995 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:52:41.372507 12995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:52:41.387187 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.381106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42383 (local address 127.0.0.1:43210)
0808 15:52:41.382085 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.382101 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.382182 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.382761 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.382767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.382793 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:41.382980 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.382989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.384376 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.384379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.385683 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.385694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.386633 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.386637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.386749 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.386790 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.386894 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.387010 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":5,"thread_start_us":102,"threads_started":1}
I20260808 15:52:41.387614 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.381013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.30.76.67:44771)
0808 15:52:41.381681 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.381692 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.382336 (+ 644us) server_negotiation.cc:407] Connection header received
0808 15:52:41.382429 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.382435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.382656 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.382822 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:41.383174 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.384472 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.385898 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.386779 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.386896 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.387025 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.387149 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.387256 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":150,"threads_started":1}
I20260808 15:52:41.388013 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.382153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40161 (local address 127.0.0.1:56752)
0808 15:52:41.382621 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.382636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.382731 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.383670 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.383673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.383693 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:41.383776 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.383782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.385211 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.385214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.386349 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.386360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.387597 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.387601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.387689 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.387713 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.387790 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.387863 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":25,"thread_start_us":103,"threads_started":1}
I20260808 15:52:41.388105 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.382580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.30.76.66:40161)
0808 15:52:41.383175 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.383188 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.383209 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:52:41.383292 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.383297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.383499 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.383701 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:41.383869 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.385060 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.386495 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.387392 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.387614 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.387764 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.387821 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.387906 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":118,"threads_started":1}
I20260808 15:52:41.387388 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.380901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44771 (local address 127.0.0.1:42616)
0808 15:52:41.381964 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.382022 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.382178 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.382853 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.382856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.382874 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:41.383045 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.383052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.384642 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.384645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.385768 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.385786 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.386895 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.386899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.387001 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.387028 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.387121 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.387200 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":50,"thread_start_us":88,"threads_started":1}
I20260808 15:52:41.388509 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.381282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.30.76.65:42383)
0808 15:52:41.381681 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.381690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.382203 (+ 513us) server_negotiation.cc:407] Connection header received
0808 15:52:41.382330 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.382336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.382558 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.383147 (+ 589us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:41.383268 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.384241 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.385829 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.386504 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.386628 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.386784 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.388219 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.388327 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":91,"threads_started":1}
I20260808 15:52:41.391325 13345 tablet_service.cc:1467] Processing CreateTablet for tablet 8367a7df2a7e4ca6b1a3236dedeb212b (DEFAULT_TABLE table=test-table [id=68c38461e0f34b729aadde6ab876133e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:41.391520 13513 tablet_service.cc:1467] Processing CreateTablet for tablet 8367a7df2a7e4ca6b1a3236dedeb212b (DEFAULT_TABLE table=test-table [id=68c38461e0f34b729aadde6ab876133e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:41.391513 13160 tablet_service.cc:1467] Processing CreateTablet for tablet 8367a7df2a7e4ca6b1a3236dedeb212b (DEFAULT_TABLE table=test-table [id=68c38461e0f34b729aadde6ab876133e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:52:41.392521 13345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8367a7df2a7e4ca6b1a3236dedeb212b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:41.392644 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8367a7df2a7e4ca6b1a3236dedeb212b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:41.392637 13160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8367a7df2a7e4ca6b1a3236dedeb212b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:52:41.398527 13631 tablet_bootstrap.cc:492] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Bootstrap starting.
I20260808 15:52:41.400106 13632 tablet_bootstrap.cc:492] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: Bootstrap starting.
I20260808 15:52:41.400755 13633 tablet_bootstrap.cc:492] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Bootstrap starting.
I20260808 15:52:41.402197 13631 tablet_bootstrap.cc:654] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:41.402350 13632 tablet_bootstrap.cc:654] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:41.402774 13633 tablet_bootstrap.cc:654] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Neither blocks nor log segments found. Creating new log.
I20260808 15:52:41.403223 13632 log.cc:824] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:41.403223 13631 log.cc:824] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:41.404211 13633 log.cc:824] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Log is configured to *not* fsync() on all Append() calls
I20260808 15:52:41.405304 13631 tablet_bootstrap.cc:492] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: No bootstrap required, opened a new log
I20260808 15:52:41.405304 13632 tablet_bootstrap.cc:492] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: No bootstrap required, opened a new log
I20260808 15:52:41.405500 13632 ts_tablet_manager.cc:1397] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 15:52:41.405500 13631 ts_tablet_manager.cc:1397] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260808 15:52:41.406201 13633 tablet_bootstrap.cc:492] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: No bootstrap required, opened a new log
I20260808 15:52:41.406420 13633 ts_tablet_manager.cc:1397] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 15:52:41.409447 13631 raft_consensus.cc:348] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.409751 13631 raft_consensus.cc:374] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:41.409817 13631 raft_consensus.cc:729] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175a90b7c7174c06818eb034e179bbcb, State: Initialized, Role: FOLLOWER
I20260808 15:52:41.409772 13632 raft_consensus.cc:348] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.409825 13633 raft_consensus.cc:348] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.410058 13632 raft_consensus.cc:374] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:41.410117 13632 raft_consensus.cc:729] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f2cd8c9f9d49e3b01cf6a854ef960a, State: Initialized, Role: FOLLOWER
I20260808 15:52:41.410135 13633 raft_consensus.cc:374] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:52:41.410207 13633 raft_consensus.cc:729] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d5c7746b24f4031a45db1e2e84844f0, State: Initialized, Role: FOLLOWER
I20260808 15:52:41.410274 13631 consensus_queue.cc:260] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.410601 13632 consensus_queue.cc:260] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.410761 13633 consensus_queue.cc:260] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.411218 13425 heartbeater.cc:503] Master 127.30.76.126:36543 was elected leader, sending a full tablet report...
I20260808 15:52:41.411357 13631 ts_tablet_manager.cc:1428] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260808 15:52:41.411515 13632 ts_tablet_manager.cc:1428] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260808 15:52:41.411851 13249 heartbeater.cc:503] Master 127.30.76.126:36543 was elected leader, sending a full tablet report...
I20260808 15:52:41.412940 13633 ts_tablet_manager.cc:1428] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 15:52:41.413084 13607 heartbeater.cc:503] Master 127.30.76.126:36543 was elected leader, sending a full tablet report...
I20260808 15:52:41.442595 13638 raft_consensus.cc:482] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:52:41.442827 13638 raft_consensus.cc:504] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.444033 13638 leader_election.cc:288] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d5c7746b24f4031a45db1e2e84844f0 (127.30.76.67:44771), 50f2cd8c9f9d49e3b01cf6a854ef960a (127.30.76.65:42383)
I20260808 15:52:41.447926 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.444563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:50687 (local address 127.30.76.65:42383)
0808 15:52:41.444693 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.444699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.444716 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:52:41.444779 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.444781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.444821 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.444902 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:41.445183 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.445730 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.446848 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.447311 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.447440 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.447492 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.447725 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.447798 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:52:41.447925 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.444279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42383 (local address 127.30.76.66:50687)
0808 15:52:41.444521 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.444538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.444660 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.444927 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.444930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.444949 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:41.445064 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.445069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.445839 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.445842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.446719 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.446728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.447440 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.447443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.447586 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.447610 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.447706 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.447798 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260808 15:52:41.448181 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.444457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:53495 (local address 127.30.76.67:44771)
0808 15:52:41.444607 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.444614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.444982 (+ 368us) server_negotiation.cc:407] Connection header received
0808 15:52:41.445130 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.445133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.445187 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.445269 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:52:41.445537 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.446092 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.447211 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.447645 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.447775 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.447823 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.448001 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.448068 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:52:41.448199 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.444279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:44771 (local address 127.30.76.66:53495)
0808 15:52:41.444887 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.444898 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:52:41.445019 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:52:41.445291 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:52:41.445294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:52:41.445309 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:52:41.445428 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.445433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.446204 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.446208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.447088 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:52:41.447097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.447771 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.447775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:52:41.447876 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.447895 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 15:52:41.447984 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 15:52:41.448048 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":224,"thread_start_us":180,"threads_started":1}
I20260808 15:52:41.448902 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8367a7df2a7e4ca6b1a3236dedeb212b" candidate_uuid: "175a90b7c7174c06818eb034e179bbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d5c7746b24f4031a45db1e2e84844f0" is_pre_election: true
I20260808 15:52:41.448900 13186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8367a7df2a7e4ca6b1a3236dedeb212b" candidate_uuid: "175a90b7c7174c06818eb034e179bbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" is_pre_election: true
I20260808 15:52:41.449213 13541 raft_consensus.cc:2454] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175a90b7c7174c06818eb034e179bbcb in term 0.
I20260808 15:52:41.449213 13186 raft_consensus.cc:2454] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175a90b7c7174c06818eb034e179bbcb in term 0.
I20260808 15:52:41.449677 13274 leader_election.cc:302] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175a90b7c7174c06818eb034e179bbcb, 3d5c7746b24f4031a45db1e2e84844f0; no voters:
I20260808 15:52:41.449899 13638 raft_consensus.cc:2781] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:52:41.450006 13638 raft_consensus.cc:482] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:52:41.450067 13638 raft_consensus.cc:3037] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:41.451073 13638 raft_consensus.cc:504] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.451442 13638 leader_election.cc:288] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [CANDIDATE]: Term 1 election: Requested vote from peers 3d5c7746b24f4031a45db1e2e84844f0 (127.30.76.67:44771), 50f2cd8c9f9d49e3b01cf6a854ef960a (127.30.76.65:42383)
I20260808 15:52:41.451794 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8367a7df2a7e4ca6b1a3236dedeb212b" candidate_uuid: "175a90b7c7174c06818eb034e179bbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d5c7746b24f4031a45db1e2e84844f0"
I20260808 15:52:41.451843 13186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8367a7df2a7e4ca6b1a3236dedeb212b" candidate_uuid: "175a90b7c7174c06818eb034e179bbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a"
I20260808 15:52:41.451926 13541 raft_consensus.cc:3037] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:41.451938 13186 raft_consensus.cc:3037] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:52:41.453223 13186 raft_consensus.cc:2454] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175a90b7c7174c06818eb034e179bbcb in term 1.
I20260808 15:52:41.453223 13541 raft_consensus.cc:2454] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175a90b7c7174c06818eb034e179bbcb in term 1.
I20260808 15:52:41.453516 13274 leader_election.cc:302] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175a90b7c7174c06818eb034e179bbcb, 3d5c7746b24f4031a45db1e2e84844f0; no voters:
I20260808 15:52:41.453697 13638 raft_consensus.cc:2781] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:52:41.453922 13638 raft_consensus.cc:686] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 1 LEADER]: Becoming Leader. State: Replica: 175a90b7c7174c06818eb034e179bbcb, State: Running, Role: LEADER
I20260808 15:52:41.454267 13638 consensus_queue.cc:237] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } }
I20260808 15:52:41.457532 12995 catalog_manager.cc:5495] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 175a90b7c7174c06818eb034e179bbcb (127.30.76.66). New cstate: current_term: 1 leader_uuid: "175a90b7c7174c06818eb034e179bbcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "175a90b7c7174c06818eb034e179bbcb" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40161 } health_report { overall_health: HEALTHY } } }
I20260808 15:52:41.483376 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.480244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.30.76.66:40161)
0808 15:52:41.480379 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 15:52:41.480382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:52:41.480395 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:52:41.480464 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:52:41.480466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:52:41.480519 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:52:41.480604 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:52:41.480831 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.481318 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.482120 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:52:41.482319 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:52:41.482393 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:52:41.482663 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:52:41.483055 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:52:41.483144 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:52:41.483207 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 15:52:41.483254 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:52:41.489151 13541 raft_consensus.cc:1261] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 1 FOLLOWER]: Refusing update from remote peer 175a90b7c7174c06818eb034e179bbcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:41.489151 13186 raft_consensus.cc:1261] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a [term 1 FOLLOWER]: Refusing update from remote peer 175a90b7c7174c06818eb034e179bbcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:52:41.489868 13642 consensus_queue.cc:1035] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f2cd8c9f9d49e3b01cf6a854ef960a" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:41.490062 13638 consensus_queue.cc:1035] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d5c7746b24f4031a45db1e2e84844f0" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:52:41.499004 13651 mvcc.cc:205] Tried to move back new op lower bound from 7316293064652218368 to 7316293064519356416. Current Snapshot: MvccSnapshot[applied={T|T < 7316293064652218368}]
I20260808 15:52:41.499218 13650 mvcc.cc:205] Tried to move back new op lower bound from 7316293064652218368 to 7316293064519356416. Current Snapshot: MvccSnapshot[applied={T|T < 7316293064652218368}]
I20260808 15:52:41.507578 12995 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:41.507745 12995 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:52:41.510293 12995 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa4f02bb61b34572a8db8993fce027a7: Sending DeleteTablet for 3 replicas of tablet 8367a7df2a7e4ca6b1a3236dedeb212b
I20260808 15:52:41.511178 13160 tablet_service.cc:1514] Processing DeleteTablet for tablet 8367a7df2a7e4ca6b1a3236dedeb212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:41 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:43210
I20260808 15:52:41.511359 13345 tablet_service.cc:1514] Processing DeleteTablet for tablet 8367a7df2a7e4ca6b1a3236dedeb212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:41 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 15:52:41.511919 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 13069
I20260808 15:52:41.511883 13513 tablet_service.cc:1514] Processing DeleteTablet for tablet 8367a7df2a7e4ca6b1a3236dedeb212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:52:41 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:42616
I20260808 15:52:41.511957 13660 tablet_replica.cc:330] T 8367a7df2a7e4ca6b1a3236dedeb212b P 50f2cd8c9f9d49e3b01cf6a854ef960a: stopping tablet replica
I20260808 15:52:41.512614 13661 tablet_replica.cc:330] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: stopping tablet replica
I20260808 15:52:41.514376 13661 raft_consensus.cc:2229] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:52:41.514987 13661 raft_consensus.cc:2258] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:41.517903 13661 ts_tablet_manager.cc:1905] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:41.518203 13662 tablet_replica.cc:330] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: stopping tablet replica
I20260808 15:52:41.518517 13662 raft_consensus.cc:2229] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:52:41.518864 13662 raft_consensus.cc:2258] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:52:41.522420 13661 ts_tablet_manager.cc:1918] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:41.522595 13661 log.cc:1196] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/8367a7df2a7e4ca6b1a3236dedeb212b
I20260808 15:52:41.523432 13661 ts_tablet_manager.cc:1939] T 8367a7df2a7e4ca6b1a3236dedeb212b P 175a90b7c7174c06818eb034e179bbcb: Deleting consensus metadata
I20260808 15:52:41.523936 13662 ts_tablet_manager.cc:1905] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:52:41.524931 12973 catalog_manager.cc:4841] TS 175a90b7c7174c06818eb034e179bbcb (127.30.76.66:40161): tablet 8367a7df2a7e4ca6b1a3236dedeb212b (table test-table [id=68c38461e0f34b729aadde6ab876133e]) successfully deleted
I20260808 15:52:41.526564 13662 ts_tablet_manager.cc:1918] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:52:41.526672 13662 log.cc:1196] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/8367a7df2a7e4ca6b1a3236dedeb212b
I20260808 15:52:41.527009 13662 ts_tablet_manager.cc:1939] T 8367a7df2a7e4ca6b1a3236dedeb212b P 3d5c7746b24f4031a45db1e2e84844f0: Deleting consensus metadata
I20260808 15:52:41.528215 12973 catalog_manager.cc:4841] TS 3d5c7746b24f4031a45db1e2e84844f0 (127.30.76.67:44771): tablet 8367a7df2a7e4ca6b1a3236dedeb212b (table test-table [id=68c38461e0f34b729aadde6ab876133e]) successfully deleted
W20260808 15:52:41.529166 12974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:42383 (error 108)
I20260808 15:52:41.529428 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 13255
I20260808 15:52:41.530357 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:52:41.529925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42383 (local address 127.0.0.1:43216)
0808 15:52:41.530147 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 15:52:41.530242 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260808 15:52:41.531177 12974 catalog_manager.cc:4584] TS 50f2cd8c9f9d49e3b01cf6a854ef960a (127.30.76.65:42383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8367a7df2a7e4ca6b1a3236dedeb212b: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42383: connect: Connection refused (error 111)
I20260808 15:52:41.538439 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 13431
I20260808 15:52:41.550549 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 12947
2026-08-08T15:52:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2522 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[13670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[13670](info): set up 2 sockets
Aug 08 15:52:41 dist-test-slave-sb2z krb5kdc[13670](info): commencing operation
krb5kdc: starting...
W20260808 15:52:42.781381 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:52:42 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:52:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/rangeradmin_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/rangeradmin_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/rangerlookup_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/rangerlookup_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
I20260808 15:52:42.950230 31025 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2MyjDc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/postgres -l logfile start
2026-08-08 15:52:45.374 UTC [13720] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 15:52:45.374 UTC [13720] LOG: listening on IPv4 address "127.30.76.84", port 57995
2026-08-08 15:52:45.377 UTC [13720] LOG: listening on Unix socket "/tmp/.s.PGSQL.57995"
2026-08-08 15:52:45.399 UTC [13723] LOG: database system was shut down at 2026-08-08 15:52:44 UTC
2026-08-08 15:52:45.402 UTC [13720] LOG: database system is ready to accept connections
I20260808 15:52:46.523859 31025 mini_postgres.cc:96] Postgres bound to 57995
2026-08-08 15:52:46.533 UTC [13737] FATAL: database "slave" does not exist
127.30.76.84:57995 - accepting connections
I20260808 15:52:46.534173 31025 mini_ranger.cc:161] Starting Ranger...
I20260808 15:52:46.550565 31025 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 15:52:46.979753 31025 mini_ranger.cc:87] Created ranger Postgres database
I20260808 15:52:46.979835 31025 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-admin
2026-08-08 15:52:47,333 [I] DB FLAVOR :POSTGRES
2026-08-08 15:52:47,334 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:52:47,334 [I] Checking connection..
2026-08-08 15:52:47,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 15:52:47,601 [I] Checking connection passed.
2026-08-08 15:52:47,602 [I] --------- Verifying version history table ---------
2026-08-08 15:52:47,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:52:47,863 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:52:47,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:52:48,126 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:52:48,126 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 15:52:48,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 15:52:48.388 UTC [13827] WARNING: there is no transaction in progress
2026-08-08 15:52:48,395 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 15:52:48,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:52:48,647 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 15:52:48,648 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:52:48,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 15:52:48,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 15:52:49,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-08 15:52:49,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:52:49,682 [I] Table x_portal_user does not exist in database ranger
2026-08-08 15:52:49,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:52:49,949 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 15:52:49,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:52:50,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:52:50,437 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 15:52:50,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 15:52:51.405 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.421 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.436 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.451 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.466 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.482 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.535 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.542 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.556 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.566 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.575 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.588 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.597 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.603 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.609 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:51.616 UTC [13981] WARNING: there is no transaction in progress
2026-08-08 15:52:52,211 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 15:52:52,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 15:52:52,466 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 15:52:52,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:52:52,707 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 15:52:52,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:52:52,969 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 15:52:52,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:52:53,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:52:53,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-08 15:52:53,724 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 15:52:53,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 15:52:53,977 [I] DB_PATCHES have already been applied
I20260808 15:52:53.983778 31025 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-admin:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2MyjDc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 15:52:53.983862 31025 mini_ranger.cc:193] Using host: 127.30.76.84
I20260808 15:52:53.988399 31025 mini_ranger.cc:227] Ranger admin URL: http://127.30.76.84:46277
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 3:52:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 15:52:55.292167 31025 mini_ranger.cc:160] Time spent starting Ranger: real 8.758s user 0.000s sys 0.012s
Aug 08, 2026 3:53:14 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:53:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 3:53:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:53:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:53:15 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 15:53:15.972110 31025 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/rangerkms_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/rangerkms_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.76.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.76.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 15:53:16 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204396, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 15:53:16.059278 31025 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 15:53:16.069487 31025 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 15:53:16.854991 31025 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 15:53:16.855096 31025 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms
I20260808 15:53:16.855206 31025 mini_ranger_kms.cc:227] Using postgres at 127.30.76.84:57995
2026-08-08 15:53:17,108 [I] DB FLAVOR :POSTGRES
2026-08-08 15:53:17,109 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:53:17,109 [I] Checking connection
2026-08-08 15:53:17,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 15:53:17,363 [I] connection success
2026-08-08 15:53:17,364 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 15:53:17,364 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 15:53:17,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 15:53:17,627 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 15:53:17,627 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:53:17,628 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 15:53:17,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:57995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 15:53:17,886 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 15:53:18.382742 31025 mini_ranger_kms.cc:310] Created kms service
I20260808 15:53:18.593151 31025 mini_ranger_kms.cc:326] Created kudu user
I20260808 15:53:18.638583 31025 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 15:53:18 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204396, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:53:19.023769 31025 mini_ranger_kms.cc:384] Added ranger policy
I20260808 15:53:19.023929 31025 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms:/tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 15:53:19.023953 31025 mini_ranger_kms.cc:240] Using host: 127.30.76.84
I20260808 15:53:19.028680 31025 mini_ranger_kms.cc:276] Ranger KMS PID: 14229
I20260808 15:53:19.028839 31025 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.76.84:55925
Aug 08, 2026 3:53:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:53:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:53:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:53:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:53:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:53:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 15:53:21 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786204401, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.76.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:53:21.734532 31025 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.675s user 0.000s sys 0.237s
I20260808 15:53:21.734685 31025 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 15:53:21.734809 31025 mini_ranger_kms.cc:401] 127.30.76.84:55925/kms/v1/keys
Aug 08 15:53:21 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204401, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.76.84@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
Aug 08 15:53:22 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204401, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.76.84@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
Aug 08 15:53:24 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:53:24.982610 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45551
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45551
--ranger_config_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:55925/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:53:25.127790 14305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:53:25.128105 14305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:53:25.128175 14305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:53:25.132198 14305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:53:25.132272 14305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:53:25.132309 14305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:53:25.132359 14305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:53:25.132397 14305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:53:25.137814 14305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.76.84:55925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45551
--ranger_config_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:53:25.138877 14305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:53:25.139991 14305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:53:25.151007 14317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:53:25.150974 14313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:53:25.151239 14305 server_base.cc:1034] running on GCE node
W20260808 15:53:25.150982 14311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:53:25.151926 14305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:53:25.153363 14305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:53:25.154548 14305 hybrid_clock.cc:648] HybridClock initialized: now 1786204405154528 us; error 38 us; skew 500 ppm
Aug 08 15:53:25 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204405, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:53:25.161022 14305 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:53:25.161944 14305 webserver.cc:460] Webserver started at http://127.30.76.126:36893/ using document root <none> and password file <none>
I20260808 15:53:25.162497 14305 fs_manager.cc:359] Metadata directory not provided
I20260808 15:53:25.162575 14305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:53:25.162806 14305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:53:25 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204405, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:53:25.220772 14305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4"
format_stamp: "Formatted at 2026-08-08 15:53:25 on dist-test-slave-sb2z"
server_key: "b5e6a1fbff136d9d34507ac395a52e43"
server_key_iv: "d3bf42b950460ece9d2c7ed6f3a825d1"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:25.221611 14305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4"
format_stamp: "Formatted at 2026-08-08 15:53:25 on dist-test-slave-sb2z"
server_key: "b5e6a1fbff136d9d34507ac395a52e43"
server_key_iv: "d3bf42b950460ece9d2c7ed6f3a825d1"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:25.226807 14305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 15:53:25.241387 14324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:25.243048 14305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:53:25.243192 14305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4"
format_stamp: "Formatted at 2026-08-08 15:53:25 on dist-test-slave-sb2z"
server_key: "b5e6a1fbff136d9d34507ac395a52e43"
server_key_iv: "d3bf42b950460ece9d2c7ed6f3a825d1"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:25.243327 14305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:53:25.268880 14305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:53:25.272460 14305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:53:25.272740 14305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:53:25.286121 14305 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45551
I20260808 15:53:25.286252 14406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45551 every 8 connection(s)
I20260808 15:53:25.287245 14305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:53:25.290565 14409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:53:25.292408 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 14305
I20260808 15:53:25.292543 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:53:25.292843 31025 external_mini_cluster.cc:1428] Setting key 9fcc8bd1d53947b71e7a50e9bf8f0469
I20260808 15:53:25.298820 14409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4: Bootstrap starting.
I20260808 15:53:25.302315 14409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4: Neither blocks nor log segments found. Creating new log.
I20260808 15:53:25.303586 14409 log.cc:824] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:53:25.306200 14409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4: No bootstrap required, opened a new log
Aug 08 15:53:25 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:53:25.311421 14409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45551 } }
I20260808 15:53:25.311757 14409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:53:25.311828 14409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e5ebdb1eae4f558343cf1a6eb89cc4, State: Initialized, Role: FOLLOWER
I20260808 15:53:25.312318 14409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45551 } }
I20260808 15:53:25.312474 14409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:53:25.312558 14409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:53:25.312696 14409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:53:25.313933 14409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45551 } }
I20260808 15:53:25.314289 14409 leader_election.cc:302] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84e5ebdb1eae4f558343cf1a6eb89cc4; no voters:
I20260808 15:53:25.314420 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:25.295141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.30.76.126:45551)
0808 15:53:25.295850 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:25.295890 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:25.297038 (+ 1148us) server_negotiation.cc:407] Connection header received
0808 15:53:25.298342 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:25.298366 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:25.298796 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:25.299238 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:53:25.300287 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:25.302085 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:25.303368 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:25.304146 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:25.304313 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:25.308312 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:53:25.308323 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:53:25.308360 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:53:25.308394 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:53:25.310923 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:25.311458 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:25.311464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:25.311470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:25.311634 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:25.311985 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:25.311988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:25.311990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:25.312516 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:53:25.312711 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:25.313116 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 15:53:25.313400 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":144,"threads_started":1}
I20260808 15:53:25.314675 14409 leader_election.cc:288] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:53:25.315100 14418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:53:25.315440 14418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [term 1 LEADER]: Becoming Leader. State: Replica: 84e5ebdb1eae4f558343cf1a6eb89cc4, State: Running, Role: LEADER
I20260808 15:53:25.315824 14418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45551 } }
I20260808 15:53:25.316143 14409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:53:25.317302 14418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45551 } } }
I20260808 15:53:25.317561 14418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:53:25.317929 14418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84e5ebdb1eae4f558343cf1a6eb89cc4. Latest consensus state: current_term: 1 leader_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e5ebdb1eae4f558343cf1a6eb89cc4" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45551 } } }
I20260808 15:53:25.318097 14418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:53:25.319795 14409 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 15:53:25.320586 14430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:53:25.330544 14430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:53:25.346632 14446 mvcc.cc:205] Tried to move back new op lower bound from 7316293244290256896 to 7316293244174692352. Current Snapshot: MvccSnapshot[applied={T|T < 7316293244290256896}]
I20260808 15:53:25.349718 14430 catalog_manager.cc:1347] Generated new cluster ID: ef0a9bc41233459498c9f9772f622b7b
I20260808 15:53:25.349792 14430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:53:25.387782 14430 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:53:25.390298 14430 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:53:25.412467 14430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4: Generated new TSK 0
I20260808 15:53:25.413693 14430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:53:26.321784 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45551
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:53:26.469085 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:53:26.469322 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:53:26.469364 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:53:26.473752 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:53:26.473811 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:53:26.473902 14480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:53:26.479405 14480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.76.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:53:26.480537 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:53:26.481585 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:53:26.491962 14487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:53:26.491954 14492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:53:26.492691 14480 server_base.cc:1034] running on GCE node
W20260808 15:53:26.492066 14489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:53:26.493960 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:53:26.494709 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:53:26.496018 14480 hybrid_clock.cc:648] HybridClock initialized: now 1786204406495987 us; error 52 us; skew 500 ppm
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:53:26.499997 14480 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:53:26.500982 14480 webserver.cc:460] Webserver started at http://127.30.76.65:40763/ using document root <none> and password file <none>
I20260808 15:53:26.501560 14480 fs_manager.cc:359] Metadata directory not provided
I20260808 15:53:26.501701 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:53:26.502007 14480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:53:26.520502 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "e349cfd4052c4eafb93a4cdf05f292ed"
format_stamp: "Formatted at 2026-08-08 15:53:26 on dist-test-slave-sb2z"
server_key: "cb0de80a37c610f9a30bb0d165554044"
server_key_iv: "7cfa7247450b94746eede1d13e59018e"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:26.521301 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "e349cfd4052c4eafb93a4cdf05f292ed"
format_stamp: "Formatted at 2026-08-08 15:53:26 on dist-test-slave-sb2z"
server_key: "cb0de80a37c610f9a30bb0d165554044"
server_key_iv: "7cfa7247450b94746eede1d13e59018e"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:26.528798 14480 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260808 15:53:26.547091 14507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:26.548995 14480 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:53:26.549203 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "e349cfd4052c4eafb93a4cdf05f292ed"
format_stamp: "Formatted at 2026-08-08 15:53:26 on dist-test-slave-sb2z"
server_key: "cb0de80a37c610f9a30bb0d165554044"
server_key_iv: "7cfa7247450b94746eede1d13e59018e"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:26.549348 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:53:26.566428 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:53:26.570396 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:53:26.570653 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:53:26.571446 14480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:53:26.572571 14480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:53:26.572638 14480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:26.572692 14480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:53:26.572719 14480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:26.592120 14480 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:40357
I20260808 15:53:26.593659 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:53:26.596493 14680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:40357 every 8 connection(s)
I20260808 15:53:26.599972 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 14480
I20260808 15:53:26.600106 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:53:26.600417 31025 external_mini_cluster.cc:1428] Setting key e127c2201dec3ad389219afb4f7f6a6e
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:53:26.615924 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:26.598045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:49655 (local address 127.30.76.126:45551)
0808 15:53:26.598338 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:26.598343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:26.601559 (+ 3216us) server_negotiation.cc:407] Connection header received
0808 15:53:26.601601 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:26.601604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:26.601649 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:26.601728 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:53:26.603589 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.604461 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.606692 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.607053 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.607117 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:26.610725 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:53:26.610731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:53:26.610737 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:53:26.610775 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:53:26.613129 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:26.613854 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:26.613859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:26.613864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:26.613966 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:26.614395 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:26.614402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:26.614405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:26.614708 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:53:26.614833 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:26.615458 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 15:53:26.615693 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":99,"threads_started":1}
I20260808 15:53:26.616814 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:26.596632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45551 (local address 127.30.76.65:49655)
0808 15:53:26.599927 (+ 3295us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:26.599984 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:26.601263 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:26.602023 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:26.602033 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:26.602552 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:53:26.603366 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:26.603392 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.604751 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.604763 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:26.606521 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:26.606536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.607289 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.607291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:26.607467 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:26.608279 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:53:26.608305 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:53:26.610509 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:53:26.613310 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:53:26.613317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:53:26.613333 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:53:26.613710 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:53:26.614080 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:53:26.614084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:53:26.614086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:53:26.614269 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:53:26.614845 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:53:26.614852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:53:26.615154 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 15:53:26.615358 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 15:53:26.615725 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3043,"thread_start_us":222,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:53:26.618192 14681 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45551
I20260808 15:53:26.618454 14681 heartbeater.cc:464] Registering TS with master...
I20260808 15:53:26.619059 14681 heartbeater.cc:511] Master 127.30.76.126:45551 requested a full tablet report, sending...
I20260808 15:53:26.621217 14343 ts_manager.cc:194] Registered new tserver with Master: e349cfd4052c4eafb93a4cdf05f292ed (127.30.76.65:40357)
I20260808 15:53:26.622959 14343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:49655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:53:26.638332 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45551
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 15:53:26.810029 14689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:53:26.810288 14689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:53:26.810362 14689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:53:26.814425 14689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:53:26.814488 14689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:53:26.814545 14689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:53:26.821019 14689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.76.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:53:26.822256 14689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:53:26.823330 14689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:53:26.835613 14699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:53:26.836097 14701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:53:26.836236 14705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:53:26.836651 14689 server_base.cc:1034] running on GCE node
I20260808 15:53:26.837203 14689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:53:26.838158 14689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:53:26.839417 14689 hybrid_clock.cc:648] HybridClock initialized: now 1786204406839377 us; error 44 us; skew 500 ppm
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:53:26.843674 14689 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:53:26.845427 14689 webserver.cc:460] Webserver started at http://127.30.76.66:37395/ using document root <none> and password file <none>
I20260808 15:53:26.846194 14689 fs_manager.cc:359] Metadata directory not provided
I20260808 15:53:26.846302 14689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:53:26.846598 14689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:53:26.877229 14689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "07c5dd8f002348a0995bf2dbc86bb315"
format_stamp: "Formatted at 2026-08-08 15:53:26 on dist-test-slave-sb2z"
server_key: "685a5b6a8690a3193cab4853d7a9b711"
server_key_iv: "7830d9e5e7e410a56f9b49474fc2bf16"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:26.878194 14689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "07c5dd8f002348a0995bf2dbc86bb315"
format_stamp: "Formatted at 2026-08-08 15:53:26 on dist-test-slave-sb2z"
server_key: "685a5b6a8690a3193cab4853d7a9b711"
server_key_iv: "7830d9e5e7e410a56f9b49474fc2bf16"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:26.885378 14689 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260808 15:53:26.903472 14716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:26.905479 14689 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:53:26.905654 14689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "07c5dd8f002348a0995bf2dbc86bb315"
format_stamp: "Formatted at 2026-08-08 15:53:26 on dist-test-slave-sb2z"
server_key: "685a5b6a8690a3193cab4853d7a9b711"
server_key_iv: "7830d9e5e7e410a56f9b49474fc2bf16"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:26.905807 14689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:53:26.935406 14689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:53:26.939775 14689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:53:26.940040 14689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:53:26.940860 14689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:53:26.942216 14689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:53:26.942320 14689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:26.942430 14689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:53:26.942497 14689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:26.974217 14689 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:39355
I20260808 15:53:26.976168 14689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:53:26.977448 14887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:39355 every 8 connection(s)
I20260808 15:53:26.983131 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 14689
I20260808 15:53:26.983259 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:53:26.983635 31025 external_mini_cluster.cc:1428] Setting key 42707140acba893316816279fd839d3b
Aug 08 15:53:26 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:53:26.997709 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:26.979683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45551 (local address 127.30.76.66:48435)
0808 15:53:26.980457 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:26.980527 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:26.982107 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:26.982894 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:26.982913 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:26.983431 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:53:26.984220 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:26.984243 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.985585 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.985588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:26.987298 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:26.987312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.988295 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.988300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:26.988572 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:26.989380 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:53:26.989408 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:53:26.991624 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:53:26.993915 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:53:26.993925 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:53:26.993976 (+ 51us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:53:26.994355 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:53:26.994706 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:53:26.994712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:53:26.994717 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:53:26.994904 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:53:26.995436 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:53:26.995446 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:53:26.995856 (+ 410us) client_negotiation.cc:769] Sending connection context
0808 15:53:26.996083 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 15:53:26.996518 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":228,"threads_started":1}
I20260808 15:53:26.998777 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:26.979173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:48435 (local address 127.30.76.126:45551)
0808 15:53:26.979349 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:26.979354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:26.980642 (+ 1288us) server_negotiation.cc:407] Connection header received
0808 15:53:26.982385 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:26.982389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:26.982448 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:26.982556 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:53:26.984420 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.985359 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.987511 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:26.987956 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:26.988042 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:26.991847 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:53:26.991852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:53:26.991855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:53:26.991882 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:53:26.993808 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:26.994495 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:26.994501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:26.994506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:26.994599 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:26.995020 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:26.995026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:26.995030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:26.995280 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:53:26.995378 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:26.998445 (+ 3067us) server_negotiation.cc:299] Negotiation successful
0808 15:53:26.998632 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 15:53:27.000746 14888 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45551
I20260808 15:53:27.001078 14888 heartbeater.cc:464] Registering TS with master...
I20260808 15:53:27.001863 14888 heartbeater.cc:511] Master 127.30.76.126:45551 requested a full tablet report, sending...
I20260808 15:53:27.003232 14343 ts_manager.cc:194] Registered new tserver with Master: 07c5dd8f002348a0995bf2dbc86bb315 (127.30.76.66:39355)
I20260808 15:53:27.004037 14343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:48435
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:53:27.034770 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45551
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:53:27.233534 14893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:53:27.233847 14893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:53:27.233980 14893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:53:27.238848 14893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:53:27.238945 14893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:53:27.239068 14893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:53:27.245399 14893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.76.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:53:27.247407 14893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:53:27.248881 14893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:53:27.260897 14900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:53:27.260839 14902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:53:27.261528 14906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:53:27.262250 14893 server_base.cc:1034] running on GCE node
I20260808 15:53:27.262645 14893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:53:27.263306 14893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:53:27.264482 14893 hybrid_clock.cc:648] HybridClock initialized: now 1786204407264442 us; error 57 us; skew 500 ppm
Aug 08 15:53:27 dist-test-slave-sb2z krb5kdc[13670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204407, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:53:27.267700 14893 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:53:27.268687 14893 webserver.cc:460] Webserver started at http://127.30.76.67:33367/ using document root <none> and password file <none>
I20260808 15:53:27.269589 14893 fs_manager.cc:359] Metadata directory not provided
I20260808 15:53:27.269690 14893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:53:27.269908 14893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:53:27 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204407, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:53:27.287024 14893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "c9f9cf0e1f1a48b192a900d540257611"
format_stamp: "Formatted at 2026-08-08 15:53:27 on dist-test-slave-sb2z"
server_key: "1b6c6ce9892d6437ae44f7e079a3a7a7"
server_key_iv: "2d67ddaca3795d17c8c7884a8d378dfc"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:27.287956 14893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "c9f9cf0e1f1a48b192a900d540257611"
format_stamp: "Formatted at 2026-08-08 15:53:27 on dist-test-slave-sb2z"
server_key: "1b6c6ce9892d6437ae44f7e079a3a7a7"
server_key_iv: "2d67ddaca3795d17c8c7884a8d378dfc"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:27.297629 14893 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.001s
I20260808 15:53:27.316931 14920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:27.318663 14893 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260808 15:53:27.318796 14893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "c9f9cf0e1f1a48b192a900d540257611"
format_stamp: "Formatted at 2026-08-08 15:53:27 on dist-test-slave-sb2z"
server_key: "1b6c6ce9892d6437ae44f7e079a3a7a7"
server_key_iv: "2d67ddaca3795d17c8c7884a8d378dfc"
server_key_version: "kuduclusterkey@0"
I20260808 15:53:27.318935 14893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:53:27.343601 14893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:53:27.348644 14893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:53:27.349002 14893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:53:27.350054 14893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:53:27.351362 14893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:53:27.351493 14893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:27.351634 14893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:53:27.351711 14893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:53:27.377151 14893 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:36409
I20260808 15:53:27.378736 14893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:53:27.381592 15076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:36409 every 8 connection(s)
I20260808 15:53:27.387840 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 14893
I20260808 15:53:27.387975 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:53:27.388345 31025 external_mini_cluster.cc:1428] Setting key 314646c3a3074e1d846eddca53898d8d
Aug 08 15:53:27 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204407, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
Aug 08 15:53:27 dist-test-slave-sb2z krb5kdc[13670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204406, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:53:27.416317 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.386388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:46919 (local address 127.30.76.126:45551)
0808 15:53:27.386557 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.386562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.392482 (+ 5920us) server_negotiation.cc:407] Connection header received
0808 15:53:27.393932 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.393938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.394001 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.394080 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:53:27.405240 (+ 11160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.406076 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.408006 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.408353 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.408471 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.411679 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:53:27.411684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:53:27.411687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:53:27.411715 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:53:27.413620 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:27.414224 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:27.414227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:27.414229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:27.414312 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:27.414697 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:27.414700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:27.414702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:27.414916 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:53:27.415021 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.416029 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.416154 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:53:27.417665 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.386973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45551 (local address 127.30.76.67:46919)
0808 15:53:27.392317 (+ 5344us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.392381 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:27.393653 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:27.403848 (+ 10195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:27.403858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:27.404325 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:53:27.405083 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.405101 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.406334 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.406337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.407860 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.407873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.408559 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.408564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.408872 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.409549 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:53:27.409571 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:53:27.411457 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:53:27.413752 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:53:27.413757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:53:27.413774 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:53:27.414109 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:53:27.414406 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:53:27.414409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:53:27.414411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:53:27.414574 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:53:27.415077 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:53:27.415084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:53:27.415380 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 15:53:27.415605 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 15:53:27.416562 (+ 957us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5082,"thread_start_us":308,"threads_started":1}
I20260808 15:53:27.418990 15077 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45551
I20260808 15:53:27.419243 15077 heartbeater.cc:464] Registering TS with master...
I20260808 15:53:27.419849 15077 heartbeater.cc:511] Master 127.30.76.126:45551 requested a full tablet report, sending...
I20260808 15:53:27.420992 14343 ts_manager.cc:194] Registered new tserver with Master: c9f9cf0e1f1a48b192a900d540257611 (127.30.76.67:36409)
I20260808 15:53:27.421710 14343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:46919
I20260808 15:53:27.426349 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:53:27.444765 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.431409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.30.76.126:45551)
0808 15:53:27.432656 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.432661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.432676 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:53:27.432717 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.432720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.432764 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.432841 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:53:27.433944 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.434733 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.437114 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.437469 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.437627 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.440369 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:53:27.440373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:53:27.440376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:53:27.440404 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:53:27.442299 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:27.442945 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:27.442949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:27.442951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:27.443043 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:53:27.443372 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:53:27.443375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:53:27.443377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:53:27.443692 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:53:27.443831 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.444371 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.444538 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1147}
I20260808 15:53:27.449226 14343 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:53:27.453616 14343 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:53:27.488741 15097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.477331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:39355 (local address 127.0.0.1:59786)
0808 15:53:27.480399 (+ 3068us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.480412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:27.480569 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:27.481451 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:27.481458 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:27.481481 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:53:27.481620 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.481627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.484362 (+ 2735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.484365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.486627 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.486639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.488149 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.488152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.488298 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.488341 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 15:53:27.488448 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:53:27.488589 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3009,"thread_start_us":143,"threads_started":1}
I20260808 15:53:27.489668 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.477819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.30.76.66:39355)
0808 15:53:27.480259 (+ 2440us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.480275 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.480673 (+ 398us) server_negotiation.cc:407] Connection header received
0808 15:53:27.480756 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.480762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.480994 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.481278 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:53:27.481760 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.484220 (+ 2460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.486875 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.487832 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.487934 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.488083 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.489354 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.489458 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2330,"thread_start_us":153,"threads_started":1}
I20260808 15:53:27.494053 14786 tablet_service.cc:1467] Processing CreateTablet for tablet 4d40bcf8cc594d9dbef7b5785d075e31 (DEFAULT_TABLE table=test-table [id=18e327f66d2844c68c8d0a2e28bf6122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:53:27.494853 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.476187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36409 (local address 127.0.0.1:54484)
0808 15:53:27.479794 (+ 3607us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.479844 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:27.479973 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:27.488313 (+ 8340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:27.488316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:27.488333 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:53:27.488438 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.488446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.490975 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.490979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.492066 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.492076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.494439 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.494443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.494557 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.494595 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 15:53:27.494644 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 15:53:27.494707 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3427,"thread_start_us":120,"threads_started":1}
I20260808 15:53:27.495668 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.476359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.30.76.67:36409)
0808 15:53:27.487756 (+ 11397us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.487786 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.487816 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:53:27.487903 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.487909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.488148 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.488594 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:53:27.488731 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.489833 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.492242 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.493005 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.495160 (+ 2155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.495307 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.495368 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.495486 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11275,"thread_start_us":133,"threads_started":1}
I20260808 15:53:27.495934 14786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d40bcf8cc594d9dbef7b5785d075e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:53:27.499047 14973 tablet_service.cc:1467] Processing CreateTablet for tablet 4d40bcf8cc594d9dbef7b5785d075e31 (DEFAULT_TABLE table=test-table [id=18e327f66d2844c68c8d0a2e28bf6122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:53:27.500319 14973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d40bcf8cc594d9dbef7b5785d075e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:53:27.507582 15102 tablet_bootstrap.cc:492] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Bootstrap starting.
I20260808 15:53:27.510425 15102 tablet_bootstrap.cc:654] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Neither blocks nor log segments found. Creating new log.
I20260808 15:53:27.512743 15102 log.cc:824] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Log is configured to *not* fsync() on all Append() calls
I20260808 15:53:27.513578 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.478668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:40357 (local address 127.0.0.1:45104)
0808 15:53:27.483731 (+ 5063us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.483745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:27.483833 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:27.490696 (+ 6863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:27.490699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:27.490720 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:53:27.490799 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.490805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.507878 (+ 17073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.507883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.509248 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.509261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.513068 (+ 3807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.513076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.513193 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.513243 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 15:53:27.513320 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:53:27.513408 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4996,"thread_start_us":135,"threads_started":1}
I20260808 15:53:27.513564 15103 tablet_bootstrap.cc:492] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Bootstrap starting.
I20260808 15:53:27.514484 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.478850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.30.76.65:40357)
0808 15:53:27.490132 (+ 11282us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.490139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.490162 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:53:27.490233 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.490238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.490415 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.490552 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:53:27.493115 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.494225 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.509455 (+ 15230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.512911 (+ 3456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.514008 (+ 1097us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.514166 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.514222 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.514317 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11143,"thread_start_us":82,"threads_started":1}
I20260808 15:53:27.518034 14578 tablet_service.cc:1467] Processing CreateTablet for tablet 4d40bcf8cc594d9dbef7b5785d075e31 (DEFAULT_TABLE table=test-table [id=18e327f66d2844c68c8d0a2e28bf6122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:53:27.518507 15102 tablet_bootstrap.cc:492] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: No bootstrap required, opened a new log
I20260808 15:53:27.518698 15102 ts_tablet_manager.cc:1397] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260808 15:53:27.520320 14578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d40bcf8cc594d9dbef7b5785d075e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:53:27.524796 15102 raft_consensus.cc:348] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.525095 15102 raft_consensus.cc:374] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:53:27.525168 15102 raft_consensus.cc:729] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c5dd8f002348a0995bf2dbc86bb315, State: Initialized, Role: FOLLOWER
I20260808 15:53:27.535899 15103 tablet_bootstrap.cc:654] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Neither blocks nor log segments found. Creating new log.
I20260808 15:53:27.536371 15102 consensus_queue.cc:260] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.537571 15103 log.cc:824] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Log is configured to *not* fsync() on all Append() calls
I20260808 15:53:27.538803 15102 ts_tablet_manager.cc:1428] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Time spent starting tablet: real 0.020s user 0.004s sys 0.004s
I20260808 15:53:27.539649 14888 heartbeater.cc:503] Master 127.30.76.126:45551 was elected leader, sending a full tablet report...
I20260808 15:53:27.541764 15103 tablet_bootstrap.cc:492] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: No bootstrap required, opened a new log
I20260808 15:53:27.542570 15103 ts_tablet_manager.cc:1397] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.000s
I20260808 15:53:27.546823 15103 raft_consensus.cc:348] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.547245 15103 raft_consensus.cc:374] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:53:27.547369 15103 raft_consensus.cc:729] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f9cf0e1f1a48b192a900d540257611, State: Initialized, Role: FOLLOWER
I20260808 15:53:27.547959 15103 consensus_queue.cc:260] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.549224 15103 ts_tablet_manager.cc:1428] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 15:53:27.549799 15105 tablet_bootstrap.cc:492] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: Bootstrap starting.
I20260808 15:53:27.551889 15077 heartbeater.cc:503] Master 127.30.76.126:45551 was elected leader, sending a full tablet report...
I20260808 15:53:27.556406 15105 tablet_bootstrap.cc:654] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: Neither blocks nor log segments found. Creating new log.
I20260808 15:53:27.557943 15105 log.cc:824] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: Log is configured to *not* fsync() on all Append() calls
I20260808 15:53:27.562225 15105 tablet_bootstrap.cc:492] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: No bootstrap required, opened a new log
I20260808 15:53:27.562495 15105 ts_tablet_manager.cc:1397] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260808 15:53:27.570722 15105 raft_consensus.cc:348] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.571415 15105 raft_consensus.cc:374] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:53:27.571717 15105 raft_consensus.cc:729] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e349cfd4052c4eafb93a4cdf05f292ed, State: Initialized, Role: FOLLOWER
I20260808 15:53:27.572703 15105 consensus_queue.cc:260] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.574746 15105 ts_tablet_manager.cc:1428] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260808 15:53:27.575304 14681 heartbeater.cc:503] Master 127.30.76.126:45551 was elected leader, sending a full tablet report...
I20260808 15:53:27.582983 15111 raft_consensus.cc:482] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:53:27.583351 15111 raft_consensus.cc:504] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.588096 15111 leader_election.cc:288] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9f9cf0e1f1a48b192a900d540257611 (127.30.76.67:36409), 07c5dd8f002348a0995bf2dbc86bb315 (127.30.76.66:39355)
I20260808 15:53:27.596463 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.586478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:53633 (local address 127.30.76.67:36409)
0808 15:53:27.586688 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.586700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.588962 (+ 2262us) server_negotiation.cc:407] Connection header received
0808 15:53:27.589723 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.589726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.589822 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.589920 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:53:27.591667 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.592434 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.594825 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.595609 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.595741 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.595826 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.596254 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.596342 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 15:53:27.597165 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.585812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36409 (local address 127.30.76.65:53633)
0808 15:53:27.588759 (+ 2947us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.588785 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:27.589095 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:27.590171 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:27.590175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:27.590201 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:53:27.590645 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.590664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.593419 (+ 2755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.593422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.594675 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.594686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.595943 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.595947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.596101 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.596125 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:53:27.596823 (+ 698us) client_negotiation.cc:240] Negotiation successful
0808 15:53:27.596899 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2790,"thread_start_us":198,"threads_started":1}
I20260808 15:53:27.598140 15003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d40bcf8cc594d9dbef7b5785d075e31" candidate_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f9cf0e1f1a48b192a900d540257611" is_pre_election: true
I20260808 15:53:27.598479 15003 raft_consensus.cc:2454] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e349cfd4052c4eafb93a4cdf05f292ed in term 0.
I20260808 15:53:27.599082 14510 leader_election.cc:302] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9f9cf0e1f1a48b192a900d540257611, e349cfd4052c4eafb93a4cdf05f292ed; no voters:
I20260808 15:53:27.599687 15112 raft_consensus.cc:2781] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:53:27.599856 15112 raft_consensus.cc:482] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:53:27.599962 15112 raft_consensus.cc:3037] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 0 FOLLOWER]: Advancing to term 1
W20260808 15:53:27.601305 14683 tablet.cc:2369] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:53:27.602511 15112 raft_consensus.cc:504] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.605048 15003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d40bcf8cc594d9dbef7b5785d075e31" candidate_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f9cf0e1f1a48b192a900d540257611"
I20260808 15:53:27.605177 15003 raft_consensus.cc:3037] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:53:27.606492 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.587990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:41527 (local address 127.30.76.66:39355)
0808 15:53:27.588147 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.588152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.589585 (+ 1433us) server_negotiation.cc:407] Connection header received
0808 15:53:27.591040 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.591043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.591094 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.591139 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:53:27.602752 (+ 11613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.603561 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.605000 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.605623 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.605743 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.605814 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.606271 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.606347 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:53:27.607015 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.587238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:39355 (local address 127.30.76.65:41527)
0808 15:53:27.589404 (+ 2166us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.589446 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:53:27.590878 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:53:27.591262 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:53:27.591265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:53:27.591280 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:53:27.591627 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.591641 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.603704 (+ 12063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.603707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.604849 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:53:27.604860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.605942 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.605953 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:53:27.606081 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.606105 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:53:27.606748 (+ 643us) client_negotiation.cc:240] Negotiation successful
0808 15:53:27.606869 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2058,"thread_start_us":202,"threads_started":1}
I20260808 15:53:27.607234 15003 raft_consensus.cc:2454] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e349cfd4052c4eafb93a4cdf05f292ed in term 1.
I20260808 15:53:27.607782 14510 leader_election.cc:302] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9f9cf0e1f1a48b192a900d540257611, e349cfd4052c4eafb93a4cdf05f292ed; no voters:
I20260808 15:53:27.607735 14795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d40bcf8cc594d9dbef7b5785d075e31" candidate_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c5dd8f002348a0995bf2dbc86bb315" is_pre_election: true
I20260808 15:53:27.607972 15111 raft_consensus.cc:2781] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:53:27.608047 14795 raft_consensus.cc:2454] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e349cfd4052c4eafb93a4cdf05f292ed in term 0.
I20260808 15:53:27.608211 15111 raft_consensus.cc:686] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [term 1 LEADER]: Becoming Leader. State: Replica: e349cfd4052c4eafb93a4cdf05f292ed, State: Running, Role: LEADER
I20260808 15:53:27.608567 15111 consensus_queue.cc:237] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } }
I20260808 15:53:27.611630 14351 catalog_manager.cc:5495] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed reported cstate change: term changed from 0 to 1, leader changed from <none> to e349cfd4052c4eafb93a4cdf05f292ed (127.30.76.65). New cstate: current_term: 1 leader_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40357 } health_report { overall_health: HEALTHY } } }
I20260808 15:53:27.612011 14795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d40bcf8cc594d9dbef7b5785d075e31" candidate_uuid: "e349cfd4052c4eafb93a4cdf05f292ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c5dd8f002348a0995bf2dbc86bb315"
I20260808 15:53:27.612121 14795 raft_consensus.cc:3037] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:53:27.614130 14795 raft_consensus.cc:2454] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e349cfd4052c4eafb93a4cdf05f292ed in term 1.
I20260808 15:53:27.614532 15112 leader_election.cc:288] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [CANDIDATE]: Term 1 election: Requested vote from peers c9f9cf0e1f1a48b192a900d540257611 (127.30.76.67:36409), 07c5dd8f002348a0995bf2dbc86bb315 (127.30.76.66:39355)
W20260808 15:53:27.637357 15078 tablet.cc:2369] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:53:27.646317 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.640428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.30.76.65:40357)
0808 15:53:27.640590 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:53:27.640595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:53:27.640622 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:53:27.640824 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:53:27.640828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:53:27.640919 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:53:27.641075 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:53:27.641582 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.642482 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.643918 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:53:27.644323 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:53:27.644439 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:53:27.644943 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:53:27.645526 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:53:27.645651 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 15:53:27.646090 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 15:53:27.646167 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 15:53:27.656174 14795 raft_consensus.cc:1261] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 1 FOLLOWER]: Refusing update from remote peer e349cfd4052c4eafb93a4cdf05f292ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:53:27.657088 15116 consensus_queue.cc:1035] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c5dd8f002348a0995bf2dbc86bb315" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 39355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:53:27.663931 15003 raft_consensus.cc:1261] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 1 FOLLOWER]: Refusing update from remote peer e349cfd4052c4eafb93a4cdf05f292ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:53:27.664654 15116 consensus_queue.cc:1035] T 4d40bcf8cc594d9dbef7b5785d075e31 P e349cfd4052c4eafb93a4cdf05f292ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9f9cf0e1f1a48b192a900d540257611" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:53:27.676805 15122 mvcc.cc:205] Tried to move back new op lower bound from 7316293253750198272 to 7316293253567488000. Current Snapshot: MvccSnapshot[applied={T|T < 7316293253750198272}]
I20260808 15:53:27.694811 14343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:53:27.694977 14343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:53:27.697634 14343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84e5ebdb1eae4f558343cf1a6eb89cc4: Sending DeleteTablet for 3 replicas of tablet 4d40bcf8cc594d9dbef7b5785d075e31
I20260808 15:53:27.698582 14973 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d40bcf8cc594d9dbef7b5785d075e31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:53:27 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:54484
I20260808 15:53:27.699615 14786 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d40bcf8cc594d9dbef7b5785d075e31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:53:27 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59786
I20260808 15:53:27.700197 14578 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d40bcf8cc594d9dbef7b5785d075e31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:53:27 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:45104
I20260808 15:53:27.700830 15123 mvcc.cc:205] Tried to move back new op lower bound from 7316293253750198272 to 7316293253567488000. Current Snapshot: MvccSnapshot[applied={T|T < 7316293253750198272 or (T in {7316293253750198272})}]
I20260808 15:53:27.701097 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 14480
I20260808 15:53:27.702100 15141 tablet_replica.cc:330] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: stopping tablet replica
I20260808 15:53:27.702507 15141 raft_consensus.cc:2229] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:53:27.702908 15141 raft_consensus.cc:2258] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:53:27.704435 15144 tablet_replica.cc:330] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: stopping tablet replica
I20260808 15:53:27.704811 15144 raft_consensus.cc:2229] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:53:27.705150 15144 raft_consensus.cc:2258] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:53:27.705562 15141 ts_tablet_manager.cc:1905] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:53:27.706856 15144 ts_tablet_manager.cc:1905] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:53:27.708667 15141 ts_tablet_manager.cc:1918] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:53:27.708838 15141 log.cc:1196] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/4d40bcf8cc594d9dbef7b5785d075e31
I20260808 15:53:27.709221 15141 ts_tablet_manager.cc:1939] T 4d40bcf8cc594d9dbef7b5785d075e31 P c9f9cf0e1f1a48b192a900d540257611: Deleting consensus metadata
I20260808 15:53:27.710510 14326 catalog_manager.cc:4841] TS c9f9cf0e1f1a48b192a900d540257611 (127.30.76.67:36409): tablet 4d40bcf8cc594d9dbef7b5785d075e31 (table test-table [id=18e327f66d2844c68c8d0a2e28bf6122]) successfully deleted
I20260808 15:53:27.710774 15144 ts_tablet_manager.cc:1918] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:53:27.710877 15144 log.cc:1196] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/4d40bcf8cc594d9dbef7b5785d075e31
I20260808 15:53:27.711190 15144 ts_tablet_manager.cc:1939] T 4d40bcf8cc594d9dbef7b5785d075e31 P 07c5dd8f002348a0995bf2dbc86bb315: Deleting consensus metadata
I20260808 15:53:27.712399 14329 catalog_manager.cc:4841] TS 07c5dd8f002348a0995bf2dbc86bb315 (127.30.76.66:39355): tablet 4d40bcf8cc594d9dbef7b5785d075e31 (table test-table [id=18e327f66d2844c68c8d0a2e28bf6122]) successfully deleted
W20260808 15:53:27.716782 14332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:40357 (error 108)
I20260808 15:53:27.718087 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 14689
I20260808 15:53:27.772513 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:53:27.772034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:40357 (local address 127.0.0.1:45126)
0808 15:53:27.772270 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:53:27.772372 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:40357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260808 15:53:27.772924 14332 catalog_manager.cc:4584] TS e349cfd4052c4eafb93a4cdf05f292ed (127.30.76.65:40357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d40bcf8cc594d9dbef7b5785d075e31: Network error: Client connection negotiation failed: client connection to 127.30.76.65:40357: connect: Connection refused (error 111)
I20260808 15:53:27.773900 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 14893
I20260808 15:53:27.817816 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 14305
I20260808 15:53:27.833451 31025 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 15:53:28.177783 31025 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 15:53:28.177939 31025 mini_ranger.cc:66] Stopping Ranger...
I20260808 15:53:28.522116 31025 mini_ranger.cc:68] Stopped Ranger
2026-08-08 15:53:28.522 UTC [13720] LOG: received smart shutdown request
2026-08-08 15:53:28.525 UTC [13720] LOG: background worker "logical replication launcher" (PID 13735) exited with exit code 1
2026-08-08 15:53:28.526 UTC [13724] LOG: shutting down
2026-08-08 15:53:28.662 UTC [13720] LOG: database system is shut down
2026-08-08T15:53:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47227 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:53:28 dist-test-slave-sb2z krb5kdc[15161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:53:28 dist-test-slave-sb2z krb5kdc[15161](info): set up 2 sockets
Aug 08 15:53:28 dist-test-slave-sb2z krb5kdc[15161](info): commencing operation
krb5kdc: starting...
W20260808 15:53:30.000656 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:53:30 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:53:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:53:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/rangeradmin_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/rangeradmin_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/rangerlookup_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/rangerlookup_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
I20260808 15:53:30.167762 31025 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2MyjDc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/postgres -l logfile start
2026-08-08 15:53:32.478 UTC [15204] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 15:53:32.479 UTC [15204] LOG: listening on IPv4 address "127.30.76.84", port 36075
2026-08-08 15:53:32.482 UTC [15204] LOG: listening on Unix socket "/tmp/.s.PGSQL.36075"
2026-08-08 15:53:32.517 UTC [15211] LOG: database system was shut down at 2026-08-08 15:53:31 UTC
2026-08-08 15:53:32.522 UTC [15204] LOG: database system is ready to accept connections
I20260808 15:53:33.637125 31025 mini_postgres.cc:96] Postgres bound to 36075
2026-08-08 15:53:33.644 UTC [15225] FATAL: database "slave" does not exist
127.30.76.84:36075 - accepting connections
I20260808 15:53:33.645332 31025 mini_ranger.cc:161] Starting Ranger...
I20260808 15:53:33.660535 31025 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 15:53:34.083940 31025 mini_ranger.cc:87] Created ranger Postgres database
I20260808 15:53:34.084010 31025 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-admin
2026-08-08 15:53:34,356 [I] DB FLAVOR :POSTGRES
2026-08-08 15:53:34,356 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:53:34,357 [I] Checking connection..
2026-08-08 15:53:34,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 15:53:34,619 [I] Checking connection passed.
2026-08-08 15:53:34,619 [I] --------- Verifying version history table ---------
2026-08-08 15:53:34,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:53:34,883 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:53:34,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:53:35,118 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:53:35,119 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 15:53:35,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 15:53:35.364 UTC [15319] WARNING: there is no transaction in progress
2026-08-08 15:53:35,371 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 15:53:35,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:53:35,634 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 15:53:35,635 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:53:35,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 15:53:35,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 15:53:36,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-08 15:53:36,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:53:36,617 [I] Table x_portal_user does not exist in database ranger
2026-08-08 15:53:36,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:53:36,872 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 15:53:36,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:53:37,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:53:37,365 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 15:53:37,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 15:53:38.258 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.273 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.288 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.301 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.316 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.335 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.387 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.393 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.406 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.415 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.424 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.438 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.448 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.453 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.460 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:38.465 UTC [15480] WARNING: there is no transaction in progress
2026-08-08 15:53:39,038 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 15:53:39,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 15:53:39,296 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 15:53:39,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:53:39,562 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 15:53:39,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:53:39,820 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 15:53:39,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:53:40,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:53:40,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-08 15:53:40,610 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 15:53:40,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 15:53:40,860 [I] DB_PATCHES have already been applied
I20260808 15:53:40.865815 31025 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-admin:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2MyjDc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 15:53:40.865877 31025 mini_ranger.cc:193] Using host: 127.30.76.84
I20260808 15:53:40.870214 31025 mini_ranger.cc:227] Ranger admin URL: http://127.30.76.84:43867
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 15:53:42.173703 31025 mini_ranger.cc:160] Time spent starting Ranger: real 8.528s user 0.003s sys 0.008s
Aug 08, 2026 3:54:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:54:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 3:54:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:54:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:54:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 15:54:04.093858 31025 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/rangerkms_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/rangerkms_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.76.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.76.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.76.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/HTTP_127.30.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 15:54:04 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204444, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 15:54:04.214540 31025 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 15:54:04.232853 31025 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 15:54:05.013485 31025 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 15:54:05.013573 31025 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms
I20260808 15:54:05.013592 31025 mini_ranger_kms.cc:227] Using postgres at 127.30.76.84:36075
2026-08-08 15:54:05,290 [I] DB FLAVOR :POSTGRES
2026-08-08 15:54:05,291 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:54:05,291 [I] Checking connection
2026-08-08 15:54:05,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 15:54:05,617 [I] connection success
2026-08-08 15:54:05,617 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 15:54:05,617 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 15:54:05,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 15:54:05,887 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 15:54:05,888 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:54:05,888 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 15:54:05,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.76.84:36075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 15:54:06,153 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 15:54:06.631992 31025 mini_ranger_kms.cc:310] Created kms service
I20260808 15:54:06.807482 31025 mini_ranger_kms.cc:326] Created kudu user
I20260808 15:54:06.851778 31025 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 15:54:06 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204444, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:07.214623 31025 mini_ranger_kms.cc:384] Added ranger policy
I20260808 15:54:07.214811 31025 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms:/tmp/dist-test-task2MyjDc/build/debug/bin/postgresql.jar:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2MyjDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 15:54:07.214844 31025 mini_ranger_kms.cc:240] Using host: 127.30.76.84
I20260808 15:54:07.221017 31025 mini_ranger_kms.cc:276] Ranger KMS PID: 15764
I20260808 15:54:07.221107 31025 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.76.84:35437
Aug 08, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:54:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:54:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 15:54:09 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786204449, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.76.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:10.018821 31025 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.804s user 0.010s sys 0.240s
I20260808 15:54:10.018956 31025 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 15:54:10.019028 31025 mini_ranger_kms.cc:401] 127.30.76.84:35437/kms/v1/keys
Aug 08 15:54:10 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204449, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.76.84@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
Aug 08 15:54:10 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204449, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.76.84@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
Aug 08 15:54:12 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:54:12.940852 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:38151
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:38151
--ranger_config_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:13.193531 15840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:13.193874 15840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:13.193977 15840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:13.198244 15840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:13.198329 15840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:13.198369 15840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:13.198424 15840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:13.198462 15840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:13.204669 15840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.76.84:35437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:38151
--ranger_config_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:38151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:13.206449 15840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:13.208230 15840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:13.221010 15846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:13.220973 15845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:13.220968 15848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:13.221658 15840 server_base.cc:1034] running on GCE node
I20260808 15:54:13.222568 15840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:13.224334 15840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:13.225672 15840 hybrid_clock.cc:648] HybridClock initialized: now 1786204453225654 us; error 77 us; skew 500 ppm
Aug 08 15:54:13 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204453, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:13.228703 15840 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:13.229985 15840 webserver.cc:460] Webserver started at http://127.30.76.126:33621/ using document root <none> and password file <none>
I20260808 15:54:13.230911 15840 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:13.230970 15840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:13.231205 15840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:54:13 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204453, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:13.279115 15840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "bcb9bd32697542c0a007c0db9e751ef8"
format_stamp: "Formatted at 2026-08-08 15:54:13 on dist-test-slave-sb2z"
server_key: "8a11c1ab8db394e50184a60b318f4149"
server_key_iv: "40b891a653c14a183037321eb1d00c60"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:13.279845 15840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "bcb9bd32697542c0a007c0db9e751ef8"
format_stamp: "Formatted at 2026-08-08 15:54:13 on dist-test-slave-sb2z"
server_key: "8a11c1ab8db394e50184a60b318f4149"
server_key_iv: "40b891a653c14a183037321eb1d00c60"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:13.285964 15840 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 15:54:13.302170 15859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:13.304230 15840 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 15:54:13.304445 15840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "bcb9bd32697542c0a007c0db9e751ef8"
format_stamp: "Formatted at 2026-08-08 15:54:13 on dist-test-slave-sb2z"
server_key: "8a11c1ab8db394e50184a60b318f4149"
server_key_iv: "40b891a653c14a183037321eb1d00c60"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:13.304596 15840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:13.329560 15840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:13.332934 15840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:13.333174 15840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:13.343554 15840 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:38151
I20260808 15:54:13.343621 15934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:38151 every 8 connection(s)
I20260808 15:54:13.344792 15840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:13.350224 15937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:13.352236 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 15840
I20260808 15:54:13.352370 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:54:13.352720 31025 external_mini_cluster.cc:1428] Setting key a03beb81a799becf2bae8c211ba56b63
I20260808 15:54:13.359781 15937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Bootstrap starting.
I20260808 15:54:13.363782 15937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Neither blocks nor log segments found. Creating new log.
Aug 08 15:54:13 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:13.365370 15937 log.cc:824] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:13.367791 15937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: No bootstrap required, opened a new log
I20260808 15:54:13.372319 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:13.354617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.30.76.126:38151)
0808 15:54:13.355212 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:13.355234 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:13.355307 (+ 73us) server_negotiation.cc:407] Connection header received
0808 15:54:13.356316 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:13.356352 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:13.356894 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:13.357264 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:13.358165 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:13.359915 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:13.361164 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:13.362039 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:13.362239 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:13.365665 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:13.365671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:13.365688 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:13.365721 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:13.368647 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:13.369247 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:13.369254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:13.369260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:13.369390 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:13.369713 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:13.369716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:13.369717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:13.370250 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:13.370494 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:13.370765 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 15:54:13.371039 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":138,"threads_started":1}
I20260808 15:54:13.372489 15937 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:13.372776 15937 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:13.372869 15937 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb9bd32697542c0a007c0db9e751ef8, State: Initialized, Role: FOLLOWER
I20260808 15:54:13.373474 15937 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:13.373799 15937 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:13.373884 15937 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:13.374060 15937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:13.375239 15937 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:13.375752 15937 leader_election.cc:302] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcb9bd32697542c0a007c0db9e751ef8; no voters:
I20260808 15:54:13.376114 15937 leader_election.cc:288] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:54:13.376194 15943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:13.376410 15943 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 LEADER]: Becoming Leader. State: Replica: bcb9bd32697542c0a007c0db9e751ef8, State: Running, Role: LEADER
I20260808 15:54:13.376863 15943 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:13.377830 15937 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:13.378189 15943 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcb9bd32697542c0a007c0db9e751ef8. Latest consensus state: current_term: 1 leader_uuid: "bcb9bd32697542c0a007c0db9e751ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } } }
I20260808 15:54:13.378364 15943 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:13.378855 15949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:13.379382 15946 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcb9bd32697542c0a007c0db9e751ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } } }
I20260808 15:54:13.379611 15946 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:13.385208 15949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:13.385677 15937 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 15:54:13.397900 15949 catalog_manager.cc:1347] Generated new cluster ID: 0626f6c53d994f58a77b0a829a4d8755
I20260808 15:54:13.398022 15949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:13.422940 15949 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:54:13.424474 15949 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:13.432776 15949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Generated new TSK 0
I20260808 15:54:13.434002 15949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:54:14.196156 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:38151
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:14.347664 16001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:14.348017 16001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:14.348119 16001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:14.354221 16001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:14.354292 16001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:14.354380 16001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:54:14.360728 16001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.76.84:35437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:14.361929 16001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:14.362967 16001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:14.373446 16016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:14.373649 16001 server_base.cc:1034] running on GCE node
W20260808 15:54:14.373538 16020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:14.373790 16014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:14.374253 16001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:14.375017 16001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:14.376230 16001 hybrid_clock.cc:648] HybridClock initialized: now 1786204454376211 us; error 35 us; skew 500 ppm
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:14.379974 16001 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:54:14.381086 16001 webserver.cc:460] Webserver started at http://127.30.76.65:39453/ using document root <none> and password file <none>
I20260808 15:54:14.381771 16001 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:14.381861 16001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:14.382079 16001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:14.403296 16001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "1f2e4c34a6ba43598fe42f7f2196ad18"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "6f8448732361248e6568329df8a97027"
server_key_iv: "9cb07d62254e28d0389011e93f490f31"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:14.404117 16001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "1f2e4c34a6ba43598fe42f7f2196ad18"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "6f8448732361248e6568329df8a97027"
server_key_iv: "9cb07d62254e28d0389011e93f490f31"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:14.409423 16001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 15:54:14.441519 16029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:14.443449 16001 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260808 15:54:14.443622 16001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "1f2e4c34a6ba43598fe42f7f2196ad18"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "6f8448732361248e6568329df8a97027"
server_key_iv: "9cb07d62254e28d0389011e93f490f31"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:14.443784 16001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:14.488674 16001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:14.492354 16001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:14.492656 16001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:14.493463 16001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:14.494731 16001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:14.494837 16001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:14.494961 16001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:14.495039 16001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:14.513386 16001 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:37181
I20260808 15:54:14.514811 16001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:54:14.515693 16214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:37181 every 8 connection(s)
I20260808 15:54:14.524255 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 16001
I20260808 15:54:14.524389 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:54:14.524716 31025 external_mini_cluster.cc:1428] Setting key 45ae6259094b0ea44f4218b7d2835a0d
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:14.538986 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:14.519291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:38151 (local address 127.30.76.65:43421)
0808 15:54:14.520591 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:14.520647 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:14.521830 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:14.522718 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:14.522727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:14.523092 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:14.523764 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:14.523784 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.524975 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.524978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:14.526315 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:14.526326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.526967 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.526973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:14.527151 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:14.530262 (+ 3111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:14.530284 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:14.532632 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:14.535026 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:14.535032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:14.535051 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:14.535775 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:14.536094 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:14.536098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:14.536100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:14.536300 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:14.536809 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:14.536816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:14.537162 (+ 346us) client_negotiation.cc:769] Sending connection context
0808 15:54:14.537429 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 15:54:14.537792 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"thread_start_us":185,"threads_started":1}
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:54:14.539813 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:14.518794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:43421 (local address 127.30.76.126:38151)
0808 15:54:14.519046 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:14.519052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:14.522138 (+ 3086us) server_negotiation.cc:407] Connection header received
0808 15:54:14.522198 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:14.522201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:14.522261 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:14.522403 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:14.523939 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.524812 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.526453 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.526787 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.526925 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:14.532850 (+ 5925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:14.532854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:14.532858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:14.532887 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:14.534870 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:14.535896 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:14.535900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:14.535902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:14.535993 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:14.536416 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:14.536420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:14.536422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:14.536655 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:14.536753 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:14.539487 (+ 2734us) server_negotiation.cc:299] Negotiation successful
0808 15:54:14.539643 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260808 15:54:14.541590 16215 heartbeater.cc:347] Connected to a master server at 127.30.76.126:38151
I20260808 15:54:14.541978 16215 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:14.543005 16215 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:14.546265 15885 ts_manager.cc:194] Registered new tserver with Master: 1f2e4c34a6ba43598fe42f7f2196ad18 (127.30.76.65:37181)
I20260808 15:54:14.548345 15885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:43421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:54:14.566468 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:38151
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:14.726933 16223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:14.727385 16223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:14.727519 16223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:14.733865 16223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:14.734025 16223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:14.734172 16223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:54:14.741916 16223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.76.84:35437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:14.743497 16223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:14.744998 16223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:14.757731 16230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:14.757774 16234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:14.758574 16232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:14.759074 16223 server_base.cc:1034] running on GCE node
I20260808 15:54:14.759559 16223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:14.760325 16223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:14.761508 16223 hybrid_clock.cc:648] HybridClock initialized: now 1786204454761444 us; error 78 us; skew 500 ppm
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:14.765193 16223 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:54:14.766350 16223 webserver.cc:460] Webserver started at http://127.30.76.66:38671/ using document root <none> and password file <none>
I20260808 15:54:14.767091 16223 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:14.767266 16223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:14.767596 16223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:14.787261 16223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "34f5c73c0d8747308871b4f8a88d224b"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "3da9abb8c517db687f20773cc61585be"
server_key_iv: "1df2688fc52dda4de0e8ef432d4ccf20"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:14.788324 16223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "34f5c73c0d8747308871b4f8a88d224b"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "3da9abb8c517db687f20773cc61585be"
server_key_iv: "1df2688fc52dda4de0e8ef432d4ccf20"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:14.794004 16223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 15:54:14.824565 16242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:14.827809 16223 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.000s
I20260808 15:54:14.828011 16223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "34f5c73c0d8747308871b4f8a88d224b"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "3da9abb8c517db687f20773cc61585be"
server_key_iv: "1df2688fc52dda4de0e8ef432d4ccf20"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:14.828224 16223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:14.868793 16223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:14.872784 16223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:14.873055 16223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:14.873855 16223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:14.875047 16223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:14.875164 16223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:14.875270 16223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:14.875344 16223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:14.893116 16223 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:32879
I20260808 15:54:14.895011 16223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:54:14.897414 16399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:32879 every 8 connection(s)
I20260808 15:54:14.900550 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 16223
I20260808 15:54:14.900701 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:54:14.901054 31025 external_mini_cluster.cc:1428] Setting key 17838192ef3df142550a5d16ec3faf94
Aug 08 15:54:14 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:14.934490 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:14.899782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56135 (local address 127.30.76.126:38151)
0808 15:54:14.899937 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:14.899942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:14.912145 (+ 12203us) server_negotiation.cc:407] Connection header received
0808 15:54:14.913546 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:14.913550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:14.913613 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:14.913670 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:14.923652 (+ 9982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.924525 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.926493 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.926875 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.926960 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:14.930176 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:14.930180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:14.930183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:14.930212 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:14.931970 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:14.932628 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:14.932632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:14.932634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:14.932721 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:14.933129 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:14.933133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:14.933134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:14.933355 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:14.933443 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:14.934169 (+ 726us) server_negotiation.cc:299] Negotiation successful
0808 15:54:14.934317 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:54:14.936430 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:14.899922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:38151 (local address 127.30.76.66:56135)
0808 15:54:14.911932 (+ 12010us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:14.912018 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:14.913251 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:14.914126 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:14.914136 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:14.914538 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:14.915248 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:14.915270 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.924726 (+ 9456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.924730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:14.926336 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:14.926352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:14.927190 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:14.927194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:14.927408 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:14.928141 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:14.928163 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:14.929965 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:14.932106 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:14.932112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:14.932129 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:14.932484 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:14.932827 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:14.932829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:14.932831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:14.933005 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:14.933531 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:14.933538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:14.933848 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 15:54:14.934976 (+ 1128us) client_negotiation.cc:240] Negotiation successful
0808 15:54:14.935330 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11685,"thread_start_us":158,"threads_started":1}
I20260808 15:54:14.937863 16401 heartbeater.cc:347] Connected to a master server at 127.30.76.126:38151
I20260808 15:54:14.938109 16401 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:14.938716 16401 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:14.939904 15885 ts_manager.cc:194] Registered new tserver with Master: 34f5c73c0d8747308871b4f8a88d224b (127.30.76.66:32879)
I20260808 15:54:14.940642 15885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:56135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:54:14.970299 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:38151
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204454, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
W20260808 15:54:15.151144 16407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:15.151583 16407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:15.151696 16407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:15.157447 16407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:15.157593 16407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:15.157724 16407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:54:15.165177 16407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.76.84:35437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:15.166831 16407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:15.168238 16407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:15.181563 16413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:15.182006 16412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:15.182200 16407 server_base.cc:1034] running on GCE node
W20260808 15:54:15.182308 16415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:15.182838 16407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:15.183624 16407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:15.184779 16407 hybrid_clock.cc:648] HybridClock initialized: now 1786204455184705 us; error 91 us; skew 500 ppm
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:15.191193 16407 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:54:15.192473 16407 webserver.cc:460] Webserver started at http://127.30.76.67:37463/ using document root <none> and password file <none>
I20260808 15:54:15.193169 16407 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:15.193243 16407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:15.193463 16407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:15.213600 16407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "234f42351148498e891e611b1922fc99"
format_stamp: "Formatted at 2026-08-08 15:54:15 on dist-test-slave-sb2z"
server_key: "b8041d3780a0b3c7757448578455d4fd"
server_key_iv: "e56e26620f6934ee077d3b5b3fd9a8f9"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:15.214646 16407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "234f42351148498e891e611b1922fc99"
format_stamp: "Formatted at 2026-08-08 15:54:15 on dist-test-slave-sb2z"
server_key: "b8041d3780a0b3c7757448578455d4fd"
server_key_iv: "e56e26620f6934ee077d3b5b3fd9a8f9"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:15.223629 16407 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.000s
I20260808 15:54:15.240859 16422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.242663 16407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:54:15.242796 16407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "234f42351148498e891e611b1922fc99"
format_stamp: "Formatted at 2026-08-08 15:54:15 on dist-test-slave-sb2z"
server_key: "b8041d3780a0b3c7757448578455d4fd"
server_key_iv: "e56e26620f6934ee077d3b5b3fd9a8f9"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:15.242930 16407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:15.255375 16407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:15.258513 16407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:15.258731 16407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:15.259320 16407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:15.260344 16407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:15.260422 16407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.260509 16407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:15.260545 16407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.272496 16407 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:36837
I20260808 15:54:15.273736 16407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:54:15.275559 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 16407
I20260808 15:54:15.275699 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:54:15.275990 31025 external_mini_cluster.cc:1428] Setting key 922e371daa8a99ed5f5e627dae7ffed7
I20260808 15:54:15.285451 16593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:36837 every 8 connection(s)
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:15.303835 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:15.287734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:39893 (local address 127.30.76.126:38151)
0808 15:54:15.287946 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:15.287951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:15.289506 (+ 1555us) server_negotiation.cc:407] Connection header received
0808 15:54:15.290965 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:15.290969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:15.291033 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:15.291095 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:15.293582 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.294404 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.296135 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.296460 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.296532 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:15.299690 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:15.299694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:15.299697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:15.299726 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:15.301446 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:15.302082 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:15.302085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:15.302087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:15.302171 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:15.302561 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:15.302564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:15.302566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:15.302773 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:15.302851 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:15.303553 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 15:54:15.303682 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 15:54:15.305809 16598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:15.287496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:38151 (local address 127.30.76.67:39893)
0808 15:54:15.289253 (+ 1757us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:15.289311 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:15.290670 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:15.292195 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:15.292207 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:15.292597 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:15.293394 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:15.293415 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.294581 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.294584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:15.295996 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:15.296008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.296733 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.296737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:15.296923 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:15.297604 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:15.297624 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:15.299325 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:15.301593 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:15.301599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:15.301615 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:15.301968 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:15.302270 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:15.302273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:15.302275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:15.302451 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:15.302916 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:15.302923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:15.303222 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 15:54:15.304263 (+ 1041us) client_negotiation.cc:240] Negotiation successful
0808 15:54:15.304609 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1515,"thread_start_us":159,"threads_started":1}
I20260808 15:54:15.307242 16594 heartbeater.cc:347] Connected to a master server at 127.30.76.126:38151
I20260808 15:54:15.307495 16594 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:15.308149 16594 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:15.309293 15885 ts_manager.cc:194] Registered new tserver with Master: 234f42351148498e891e611b1922fc99 (127.30.76.67:36837)
I20260808 15:54:15.309988 15885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:39893
I20260808 15:54:15.313361 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:54:15.314216 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 16001
I20260808 15:54:15.338663 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 16223
I20260808 15:54:15.348724 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 16407
I20260808 15:54:15.361385 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 15840
I20260808 15:54:15.374106 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:38151
--webserver_interface=127.30.76.126
--webserver_port=33621
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:38151
--ranger_config_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:15.487040 16601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:15.487287 16601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:15.487325 16601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:15.492298 16601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:15.492377 16601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:15.492412 16601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:15.492456 16601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:15.492491 16601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:15.498111 16601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:38151
--ranger_config_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:38151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:15.499200 16601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:15.500579 16601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:15.509501 16611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:15.509502 16608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:15.509534 16606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:15.509893 16601 server_base.cc:1034] running on GCE node
I20260808 15:54:15.510520 16601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:15.511695 16601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:15.512877 16601 hybrid_clock.cc:648] HybridClock initialized: now 1786204455512855 us; error 53 us; skew 500 ppm
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:15.515830 16601 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:15.516723 16601 webserver.cc:460] Webserver started at http://127.30.76.126:33621/ using document root <none> and password file <none>
I20260808 15:54:15.517215 16601 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:15.517283 16601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:15.588778 16601 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:54:15.591171 16620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.592609 16601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:54:15.592756 16601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "bcb9bd32697542c0a007c0db9e751ef8"
format_stamp: "Formatted at 2026-08-08 15:54:13 on dist-test-slave-sb2z"
server_key: "8a11c1ab8db394e50184a60b318f4149"
server_key_iv: "40b891a653c14a183037321eb1d00c60"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:15.593292 16601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:15.611361 16601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:15.614460 16601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:15.614684 16601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:15.622536 16601 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:38151
I20260808 15:54:15.622568 16699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:38151 every 8 connection(s)
I20260808 15:54:15.623729 16601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:15.628320 16700 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:54:15.629879 16700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Bootstrap starting.
I20260808 15:54:15.632472 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 16601
I20260808 15:54:15.632942 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:37181
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:38151
--webserver_port=39453
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:15.647269 16700 log.cc:824] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:15.655581 16700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:54:15.656033 16700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Bootstrap complete.
I20260808 15:54:15.661468 16700 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:15.661947 16700 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb9bd32697542c0a007c0db9e751ef8, State: Initialized, Role: FOLLOWER
I20260808 15:54:15.662494 16700 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:15.662643 16700 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:15.662715 16700 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:15.662810 16700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:54:15.665316 16700 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:15.665664 16700 leader_election.cc:302] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcb9bd32697542c0a007c0db9e751ef8; no voters:
I20260808 15:54:15.665983 16700 leader_election.cc:288] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:54:15.666108 16707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:54:15.666378 16707 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [term 2 LEADER]: Becoming Leader. State: Replica: bcb9bd32697542c0a007c0db9e751ef8, State: Running, Role: LEADER
I20260808 15:54:15.666723 16707 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } }
I20260808 15:54:15.667179 16700 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:15.668273 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcb9bd32697542c0a007c0db9e751ef8. Latest consensus state: current_term: 2 leader_uuid: "bcb9bd32697542c0a007c0db9e751ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } } }
I20260808 15:54:15.668385 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:15.668730 16710 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bcb9bd32697542c0a007c0db9e751ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb9bd32697542c0a007c0db9e751ef8" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 38151 } } }
I20260808 15:54:15.668833 16710 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:15.669662 16719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:15.669878 16700 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 15:54:15.676216 16719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:15.676978 16719 catalog_manager.cc:1259] Loaded cluster ID: 0626f6c53d994f58a77b0a829a4d8755
I20260808 15:54:15.677078 16719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:15.679754 16719 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:15.680517 16719 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Loaded TSK: 0
I20260808 15:54:15.682000 16719 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:54:15.763840 16704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:15.764104 16704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:15.764194 16704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:15.768903 16704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:15.768985 16704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:15.769104 16704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:54:15.774506 16704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:37181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=39453
--tserver_master_addrs=127.30.76.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:15.775866 16704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:15.777311 16704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:15.788897 16759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:15.788882 16757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:15.789717 16704 server_base.cc:1034] running on GCE node
W20260808 15:54:15.789791 16753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:15.790161 16704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:15.790838 16704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:15.792007 16704 hybrid_clock.cc:648] HybridClock initialized: now 1786204455791965 us; error 57 us; skew 500 ppm
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:15.794950 16704 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:54:15.795995 16704 webserver.cc:460] Webserver started at http://127.30.76.65:39453/ using document root <none> and password file <none>
I20260808 15:54:15.796617 16704 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:15.796731 16704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:15.820597 16704 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.002s
I20260808 15:54:15.823390 16771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.824738 16704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 15:54:15.824880 16704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "1f2e4c34a6ba43598fe42f7f2196ad18"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "6f8448732361248e6568329df8a97027"
server_key_iv: "9cb07d62254e28d0389011e93f490f31"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:15.825440 16704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:15.839936 16704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:15.843425 16704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:15.843672 16704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:15.844420 16704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:15.845320 16704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:15.845392 16704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.845487 16704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:15.845544 16704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:15.856720 16704 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:37181
I20260808 15:54:15.857710 16704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:54:15.862556 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 16704
I20260808 15:54:15.863171 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:32879
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:38151
--webserver_port=38671
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:15.865398 16937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:37181 every 8 connection(s)
Aug 08 15:54:15 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204455, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:15.901242 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:15.868599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:38151 (local address 127.30.76.65:46569)
0808 15:54:15.875858 (+ 7259us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:15.875935 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:15.877096 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:15.879461 (+ 2365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:15.879470 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:15.879899 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:15.880714 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:15.880740 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.889420 (+ 8680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.889425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:15.890817 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:15.890829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.892020 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.892024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:15.892247 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:15.892966 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:15.892990 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:15.894816 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:15.897782 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:15.897788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:15.897800 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:15.898124 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:15.898530 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:15.898534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:15.898536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:15.898693 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:15.899388 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:15.899395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:15.899688 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 15:54:15.899876 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 15:54:15.900190 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7001,"thread_start_us":159,"threads_started":1}
I20260808 15:54:15.903682 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:15.870083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:46569 (local address 127.30.76.126:38151)
0808 15:54:15.877648 (+ 7565us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:15.877671 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:15.877717 (+ 46us) server_negotiation.cc:407] Connection header received
0808 15:54:15.878038 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:15.878053 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:15.878523 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:15.878900 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:15.887671 (+ 8771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.889236 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.890982 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:15.891827 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:15.895035 (+ 3208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:15.895234 (+ 199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:15.895240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:15.895256 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:15.895279 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:15.897571 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:15.898284 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:15.898290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:15.898296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:15.898424 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:15.898810 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:15.898813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:15.898815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:15.899252 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:15.902348 (+ 3096us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:15.902499 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 15:54:15.902777 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7389,"thread_start_us":111,"threads_started":1}
I20260808 15:54:15.906661 16938 heartbeater.cc:347] Connected to a master server at 127.30.76.126:38151
I20260808 15:54:15.907100 16938 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:15.907716 16938 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
W20260808 15:54:15.910072 16938 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:38151 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:54:15.910212 16938 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:15.910398 16938 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:15.911427 16938 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:15.911634 16938 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
W20260808 15:54:15.912544 16938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:54:16.016175 16940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:16.016515 16940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:16.016580 16940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:16.022624 16940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:16.022709 16940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:16.022795 16940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:54:16.029729 16940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:32879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=38671
--tserver_master_addrs=127.30.76.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:16.030987 16940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:16.032174 16940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:16.042362 16958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:16.042538 16955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:16.042820 16940 server_base.cc:1034] running on GCE node
W20260808 15:54:16.043111 16956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:16.043664 16940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:16.044440 16940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:16.045682 16940 hybrid_clock.cc:648] HybridClock initialized: now 1786204456045644 us; error 48 us; skew 500 ppm
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:16.049383 16940 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:54:16.050534 16940 webserver.cc:460] Webserver started at http://127.30.76.66:38671/ using document root <none> and password file <none>
I20260808 15:54:16.051226 16940 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:16.051339 16940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:16.069525 16940 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:54:16.072816 16968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:16.074486 16940 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260808 15:54:16.074620 16940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "34f5c73c0d8747308871b4f8a88d224b"
format_stamp: "Formatted at 2026-08-08 15:54:14 on dist-test-slave-sb2z"
server_key: "3da9abb8c517db687f20773cc61585be"
server_key_iv: "1df2688fc52dda4de0e8ef432d4ccf20"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:16.075268 16940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:16.084416 16940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:16.087211 16940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:16.087405 16940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:16.088040 16940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:16.088897 16940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:16.088943 16940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:16.088984 16940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:16.089041 16940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:16.100467 16940 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:32879
I20260808 15:54:16.100555 17131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:32879 every 8 connection(s)
I20260808 15:54:16.101590 16940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:54:16.104913 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 16940
I20260808 15:54:16.105444 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.76.84:35437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:36837
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:38151
--webserver_port=37463
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:16.118731 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:16.103407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:53057 (local address 127.30.76.126:38151)
0808 15:54:16.103602 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:16.103607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:16.107737 (+ 4130us) server_negotiation.cc:407] Connection header received
0808 15:54:16.109067 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:16.109070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:16.109122 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:16.109239 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:16.110312 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.110837 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.112135 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.112362 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.112486 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:16.114923 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:16.114929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:16.114936 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:16.114960 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:16.116603 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:16.117130 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:16.117134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:16.117136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:16.117211 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:16.117625 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:16.117629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:16.117630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:16.117823 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:16.117928 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:16.118413 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 15:54:16.118556 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 15:54:16.119608 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:16.103894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:38151 (local address 127.30.76.66:53057)
0808 15:54:16.104477 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:16.104537 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:16.108814 (+ 4277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:16.109427 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:16.109433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:16.109703 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:16.110162 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:16.110175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.110961 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.110965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:16.111995 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:16.112005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.112655 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.112659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:16.112828 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:16.113418 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:16.113434 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:16.114756 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:16.116733 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:16.116739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:16.116752 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:16.117008 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:16.117336 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:16.117339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:16.117340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:16.117522 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:16.117928 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:16.117934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:16.118183 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 15:54:16.118367 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 15:54:16.118677 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":155,"threads_started":1}
I20260808 15:54:16.120744 17132 heartbeater.cc:347] Connected to a master server at 127.30.76.126:38151
I20260808 15:54:16.120985 17132 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.121529 17132 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
W20260808 15:54:16.122977 17132 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:38151 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:54:16.123107 17132 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.123268 17132 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:16.124183 17132 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.124394 17132 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
W20260808 15:54:16.125198 17132 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:54:16.249145 17138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:16.249402 17138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:16.249519 17138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:16.254101 17138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:16.254180 17138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:16.254285 17138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:54:16.259989 17138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.76.84:35437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:36837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=37463
--tserver_master_addrs=127.30.76.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:16.261013 17138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:16.262002 17138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:16.274712 17148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:16.274724 17146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:16.275385 17138 server_base.cc:1034] running on GCE node
W20260808 15:54:16.275624 17145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:16.276118 17138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:16.276860 17138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:16.278048 17138 hybrid_clock.cc:648] HybridClock initialized: now 1786204456278012 us; error 46 us; skew 500 ppm
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:16.281172 17138 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:54:16.282147 17138 webserver.cc:460] Webserver started at http://127.30.76.67:37463/ using document root <none> and password file <none>
I20260808 15:54:16.282876 17138 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:16.283036 17138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:16.300212 17138 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:54:16.302789 17156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:16.304416 17138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:54:16.304559 17138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "234f42351148498e891e611b1922fc99"
format_stamp: "Formatted at 2026-08-08 15:54:15 on dist-test-slave-sb2z"
server_key: "b8041d3780a0b3c7757448578455d4fd"
server_key_iv: "e56e26620f6934ee077d3b5b3fd9a8f9"
server_key_version: "kuduclusterkey@0"
I20260808 15:54:16.305114 17138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:16.321048 17138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:16.324281 17138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:16.324486 17138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:16.325094 17138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:16.326009 17138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:16.326057 17138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:16.326097 17138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:16.326118 17138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:16.337919 17138 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:36837
I20260808 15:54:16.337980 17324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:36837 every 8 connection(s)
I20260808 15:54:16.338991 17138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:54:16.346439 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 17138
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:16.356258 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:16.340888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:54945 (local address 127.30.76.126:38151)
0808 15:54:16.341049 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:16.341056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:16.342208 (+ 1152us) server_negotiation.cc:407] Connection header received
0808 15:54:16.343683 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:16.343687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:16.343747 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:16.343810 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:16.345604 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.346400 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.348382 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.348728 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.348813 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:16.351935 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:16.351939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:16.351942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:16.351970 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:16.353750 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:16.354420 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:16.354424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:16.354426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:16.354523 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:16.354958 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:16.354962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:16.354964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:16.355191 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:16.355280 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:16.355958 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 15:54:16.356085 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 15:54:16.358233 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:16.341303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:38151 (local address 127.30.76.67:54945)
0808 15:54:16.342037 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:16.342097 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:16.343370 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:16.344254 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:16.344263 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:16.344728 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:16.345422 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:16.345445 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.346633 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.346636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:16.348231 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:16.348243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.349017 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.349021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:16.349232 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:16.349896 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:16.349920 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:16.351744 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:16.353898 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:16.353903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:16.353918 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:16.354289 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:16.354647 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:16.354650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:16.354652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:16.354834 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:16.355355 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:16.355363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:16.355688 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 15:54:16.356811 (+ 1123us) client_negotiation.cc:240] Negotiation successful
0808 15:54:16.357215 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":106,"threads_started":1}
I20260808 15:54:16.360780 17325 heartbeater.cc:347] Connected to a master server at 127.30.76.126:38151
I20260808 15:54:16.361109 17325 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.361737 17325 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
W20260808 15:54:16.363399 17325 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:38151 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:54:16.363606 17325 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.363834 17325 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:16.364715 17325 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.364897 17325 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
W20260808 15:54:16.365679 17325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:54:16.373203 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:16.347422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.30.76.126:38151)
0808 15:54:16.351099 (+ 3677us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:16.351104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:16.351117 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:54:16.351163 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:16.351165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:16.351205 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:16.351287 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:16.364666 (+ 13379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.365605 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.366969 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:16.367353 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:16.367462 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:16.369390 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:16.369394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:16.369399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:16.369427 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:16.371196 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:16.371739 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:16.371745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:16.371749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:16.371847 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:16.372205 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:16.372208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:16.372210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:16.372440 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:16.372650 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:16.372884 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 15:54:16.373015 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3606,"thread_start_us":119,"threads_started":1}
Aug 08 15:54:16 dist-test-slave-sb2z krb5kdc[15161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:16.912778 16938 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:16.913130 16938 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:16.914444 16641 ts_manager.cc:194] Registered new tserver with Master: 1f2e4c34a6ba43598fe42f7f2196ad18 (127.30.76.65:37181)
I20260808 15:54:16.916307 16641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:46569
I20260808 15:54:17.125478 17132 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:17.125785 17132 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:17.126669 16641 ts_manager.cc:194] Registered new tserver with Master: 34f5c73c0d8747308871b4f8a88d224b (127.30.76.66:32879)
I20260808 15:54:17.127480 16641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:53057
Aug 08 15:54:17 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
I20260808 15:54:17.365924 17325 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:17.366246 17325 heartbeater.cc:511] Master 127.30.76.126:38151 requested a full tablet report, sending...
I20260808 15:54:17.367074 16641 ts_manager.cc:194] Registered new tserver with Master: 234f42351148498e891e611b1922fc99 (127.30.76.67:36837)
I20260808 15:54:17.367940 16641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:54945
I20260808 15:54:17.369079 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:54:17.381177 17355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.370984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.30.76.126:38151)
0808 15:54:17.371238 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.371243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.371392 (+ 149us) server_negotiation.cc:407] Connection header received
0808 15:54:17.371496 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.371499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.371580 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.371685 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:17.372542 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.373381 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.374553 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.374911 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.375007 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.376893 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:17.376897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:17.376900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:17.376926 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:17.378742 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:17.379325 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:17.379329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:17.379331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:17.379417 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:17.379734 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:17.379737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:17.379739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:17.380012 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:17.380145 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.380850 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.380991 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260808 15:54:17.385629 16641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:54:17.390822 16641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:54:17.414381 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.405754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:32879 (local address 127.0.0.1:44514)
0808 15:54:17.408439 (+ 2685us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.408453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:17.408600 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:17.409891 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:17.409894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:17.409914 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:17.410002 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.410008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.411439 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.411443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.412643 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.412656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.413919 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.413923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.414017 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.414041 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:54:17.414119 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:54:17.414205 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2617,"thread_start_us":97,"threads_started":1}
I20260808 15:54:17.414527 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.406319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.30.76.67:36837)
0808 15:54:17.407034 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.407042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.407689 (+ 647us) server_negotiation.cc:407] Connection header received
0808 15:54:17.407774 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.407780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.407992 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.408175 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:17.408484 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.409617 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.411974 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.412785 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.412886 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.413723 (+ 837us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.414292 (+ 569us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.414391 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":122,"threads_started":1}
I20260808 15:54:17.414882 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.405427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36837 (local address 127.0.0.1:48292)
0808 15:54:17.407322 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.407373 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:17.407510 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:17.408178 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:17.408185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:17.408211 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:17.408345 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.408353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.409756 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.409760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.411835 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.411846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.413784 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.413787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.413896 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.413932 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:54:17.414038 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 15:54:17.414150 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1706,"thread_start_us":118,"threads_started":1}
I20260808 15:54:17.415609 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.406273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.30.76.66:32879)
0808 15:54:17.409221 (+ 2948us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.409229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.409253 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:54:17.409326 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.409331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.409518 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.409651 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:17.410182 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.411299 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.412796 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.413604 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.413712 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.413850 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.415352 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.415448 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2822,"thread_start_us":97,"threads_started":1}
I20260808 15:54:17.418426 17232 tablet_service.cc:1467] Processing CreateTablet for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 (DEFAULT_TABLE table=test-table [id=72cdddabc4764071ac56aff3bc53a7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:17.419768 17232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:17.421703 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.406697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:37181 (local address 127.0.0.1:43660)
0808 15:54:17.408866 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.408881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:17.408971 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:17.417224 (+ 8253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:17.417228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:17.417250 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:17.417341 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.417348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.418931 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.418935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.420141 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.420153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.421308 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.421312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.421401 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.421430 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 15:54:17.421484 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 15:54:17.421557 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2100,"thread_start_us":93,"threads_started":1}
I20260808 15:54:17.422453 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.407371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.30.76.65:37181)
0808 15:54:17.416727 (+ 9356us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.416735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.416759 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:54:17.416835 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.416840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.417039 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.417501 (+ 462us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:17.417650 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.418787 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.420304 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.421155 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.421987 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.422150 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.422206 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.422300 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9245,"thread_start_us":71,"threads_started":1}
I20260808 15:54:17.425444 16835 tablet_service.cc:1467] Processing CreateTablet for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 (DEFAULT_TABLE table=test-table [id=72cdddabc4764071ac56aff3bc53a7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:17.425522 17035 tablet_service.cc:1467] Processing CreateTablet for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 (DEFAULT_TABLE table=test-table [id=72cdddabc4764071ac56aff3bc53a7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:17.426507 17035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:17.426514 16835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:17.436064 17368 tablet_bootstrap.cc:492] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: Bootstrap starting.
I20260808 15:54:17.438581 17368 tablet_bootstrap.cc:654] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:17.439518 17368 log.cc:824] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:17.442312 17368 tablet_bootstrap.cc:492] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: No bootstrap required, opened a new log
I20260808 15:54:17.442520 17368 ts_tablet_manager.cc:1397] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:54:17.444211 17371 tablet_bootstrap.cc:492] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Bootstrap starting.
I20260808 15:54:17.446769 17368 raft_consensus.cc:348] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.447173 17368 raft_consensus.cc:374] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:17.447297 17368 raft_consensus.cc:729] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2e4c34a6ba43598fe42f7f2196ad18, State: Initialized, Role: FOLLOWER
I20260808 15:54:17.447669 17371 tablet_bootstrap.cc:654] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:17.447896 17368 consensus_queue.cc:260] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.448392 17372 tablet_bootstrap.cc:492] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Bootstrap starting.
I20260808 15:54:17.448621 17371 log.cc:824] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:17.449076 17368 ts_tablet_manager.cc:1428] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:54:17.449632 16938 heartbeater.cc:503] Master 127.30.76.126:38151 was elected leader, sending a full tablet report...
I20260808 15:54:17.450866 17372 tablet_bootstrap.cc:654] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:17.451891 17372 log.cc:824] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:17.455660 17372 tablet_bootstrap.cc:492] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: No bootstrap required, opened a new log
I20260808 15:54:17.455865 17372 ts_tablet_manager.cc:1397] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260808 15:54:17.456223 17371 tablet_bootstrap.cc:492] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: No bootstrap required, opened a new log
I20260808 15:54:17.456401 17371 ts_tablet_manager.cc:1397] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.003s
I20260808 15:54:17.459862 17372 raft_consensus.cc:348] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.460114 17372 raft_consensus.cc:374] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:17.460161 17372 raft_consensus.cc:729] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f5c73c0d8747308871b4f8a88d224b, State: Initialized, Role: FOLLOWER
I20260808 15:54:17.460217 17371 raft_consensus.cc:348] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.460498 17371 raft_consensus.cc:374] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:17.460613 17371 raft_consensus.cc:729] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234f42351148498e891e611b1922fc99, State: Initialized, Role: FOLLOWER
I20260808 15:54:17.460808 17372 consensus_queue.cc:260] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.461227 17371 consensus_queue.cc:260] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.461922 17372 ts_tablet_manager.cc:1428] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 15:54:17.462304 17371 ts_tablet_manager.cc:1428] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 15:54:17.462332 17132 heartbeater.cc:503] Master 127.30.76.126:38151 was elected leader, sending a full tablet report...
I20260808 15:54:17.462710 17325 heartbeater.cc:503] Master 127.30.76.126:38151 was elected leader, sending a full tablet report...
Aug 08 15:54:17 dist-test-slave-sb2z krb5kdc[15161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for HTTP/127.30.76.84@KRBTEST.COM
W20260808 15:54:17.591854 17326 tablet.cc:2369] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:54:17.603940 17135 tablet.cc:2369] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:54:17.618026 16939 tablet.cc:2369] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:54:17.719758 17378 raft_consensus.cc:482] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:17.719990 17378 raft_consensus.cc:504] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.721374 17378 leader_election.cc:288] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 234f42351148498e891e611b1922fc99 (127.30.76.67:36837), 1f2e4c34a6ba43598fe42f7f2196ad18 (127.30.76.65:37181)
I20260808 15:54:17.726032 17394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.721634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36837 (local address 127.30.76.66:33577)
0808 15:54:17.722127 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.722140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:17.722250 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:17.722488 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:17.722491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:17.722535 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:17.722822 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.722829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.723900 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.723903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.724876 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.724885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.725518 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.725521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.725648 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.725699 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 15:54:17.725779 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 15:54:17.725850 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":68,"spinlock_wait_cycles":9472,"thread_start_us":134,"threads_started":1}
I20260808 15:54:17.726040 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.722110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:33577 (local address 127.30.76.67:36837)
0808 15:54:17.722270 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.722275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.722289 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:54:17.722335 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.722338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.722379 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.722465 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:17.722962 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.723783 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.725003 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.725395 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.725524 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.725583 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.725827 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.725914 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:54:17.726655 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e74beed6ec4d11a3d7183f3e8ac1a0" candidate_uuid: "34f5c73c0d8747308871b4f8a88d224b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234f42351148498e891e611b1922fc99" is_pre_election: true
I20260808 15:54:17.726876 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.721785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:36893 (local address 127.30.76.65:37181)
0808 15:54:17.721967 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.721974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.722140 (+ 166us) server_negotiation.cc:407] Connection header received
0808 15:54:17.722317 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.722321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.722374 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.722465 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:17.723619 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.724264 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.725713 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.726308 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.726434 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.726480 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.726692 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.726760 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 15:54:17.726979 17234 raft_consensus.cc:2454] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f5c73c0d8747308871b4f8a88d224b in term 0.
I20260808 15:54:17.727291 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.721642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:37181 (local address 127.30.76.66:36893)
0808 15:54:17.721965 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.721979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:17.722172 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:17.722488 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:17.722491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:17.722534 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:17.722822 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.722829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.724402 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.724406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.725575 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:17.725588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.726425 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.726429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:17.726526 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.726559 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:54:17.727021 (+ 462us) client_negotiation.cc:240] Negotiation successful
0808 15:54:17.727095 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":105,"spinlock_wait_cycles":33792,"thread_start_us":111,"threads_started":1}
I20260808 15:54:17.727487 16973 leader_election.cc:302] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234f42351148498e891e611b1922fc99, 34f5c73c0d8747308871b4f8a88d224b; no voters:
I20260808 15:54:17.727836 17378 raft_consensus.cc:2781] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:54:17.727941 17378 raft_consensus.cc:482] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:54:17.728000 17378 raft_consensus.cc:3037] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:17.727943 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e74beed6ec4d11a3d7183f3e8ac1a0" candidate_uuid: "34f5c73c0d8747308871b4f8a88d224b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" is_pre_election: true
I20260808 15:54:17.728217 16866 raft_consensus.cc:2454] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f5c73c0d8747308871b4f8a88d224b in term 0.
I20260808 15:54:17.729326 17378 raft_consensus.cc:504] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.729704 17378 leader_election.cc:288] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [CANDIDATE]: Term 1 election: Requested vote from peers 234f42351148498e891e611b1922fc99 (127.30.76.67:36837), 1f2e4c34a6ba43598fe42f7f2196ad18 (127.30.76.65:37181)
I20260808 15:54:17.730098 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e74beed6ec4d11a3d7183f3e8ac1a0" candidate_uuid: "34f5c73c0d8747308871b4f8a88d224b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18"
I20260808 15:54:17.730221 16866 raft_consensus.cc:3037] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:17.731482 16866 raft_consensus.cc:2454] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f5c73c0d8747308871b4f8a88d224b in term 1.
I20260808 15:54:17.731729 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e74beed6ec4d11a3d7183f3e8ac1a0" candidate_uuid: "34f5c73c0d8747308871b4f8a88d224b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234f42351148498e891e611b1922fc99"
I20260808 15:54:17.731808 16970 leader_election.cc:302] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2e4c34a6ba43598fe42f7f2196ad18, 34f5c73c0d8747308871b4f8a88d224b; no voters:
I20260808 15:54:17.731865 17234 raft_consensus.cc:3037] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:17.731984 17378 raft_consensus.cc:2781] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:17.732259 17378 raft_consensus.cc:686] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 1 LEADER]: Becoming Leader. State: Replica: 34f5c73c0d8747308871b4f8a88d224b, State: Running, Role: LEADER
I20260808 15:54:17.732610 17378 consensus_queue.cc:237] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } }
I20260808 15:54:17.733346 17234 raft_consensus.cc:2454] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f5c73c0d8747308871b4f8a88d224b in term 1.
I20260808 15:54:17.735987 16639 catalog_manager.cc:5495] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b reported cstate change: term changed from 0 to 1, leader changed from <none> to 34f5c73c0d8747308871b4f8a88d224b (127.30.76.66). New cstate: current_term: 1 leader_uuid: "34f5c73c0d8747308871b4f8a88d224b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f5c73c0d8747308871b4f8a88d224b" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260808 15:54:17.790418 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.786091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44526 (local address 127.30.76.66:32879)
0808 15:54:17.786257 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:17.786262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:17.786280 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:54:17.786393 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:17.786396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:17.786468 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:17.786590 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:54:17.786812 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.787651 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.788569 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:17.788907 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:17.789024 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:17.789399 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:54:17.789974 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:54:17.790102 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:17.790191 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 15:54:17.790253 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:54:17.796842 16866 raft_consensus.cc:1261] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 1f2e4c34a6ba43598fe42f7f2196ad18 [term 1 FOLLOWER]: Refusing update from remote peer 34f5c73c0d8747308871b4f8a88d224b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:17.797533 17378 consensus_queue.cc:1035] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f2e4c34a6ba43598fe42f7f2196ad18" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 37181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:17.798671 17234 raft_consensus.cc:1261] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 1 FOLLOWER]: Refusing update from remote peer 34f5c73c0d8747308871b4f8a88d224b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:17.799371 17378 consensus_queue.cc:1035] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [LEADER]: Connected to new peer: Peer: permanent_uuid: "234f42351148498e891e611b1922fc99" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:17.807597 17399 mvcc.cc:205] Tried to move back new op lower bound from 7316293459128086528 to 7316293458876145664. Current Snapshot: MvccSnapshot[applied={T|T < 7316293459128086528}]
I20260808 15:54:17.808144 17402 mvcc.cc:205] Tried to move back new op lower bound from 7316293459128086528 to 7316293458876145664. Current Snapshot: MvccSnapshot[applied={T|T < 7316293459128086528}]
I20260808 15:54:17.819819 16639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:17.820022 16639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:17.822564 16639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcb9bd32697542c0a007c0db9e751ef8: Sending DeleteTablet for 3 replicas of tablet a4e74beed6ec4d11a3d7183f3e8ac1a0
I20260808 15:54:17.823377 16835 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:17 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:43660
I20260808 15:54:17.823369 17232 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:17 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:48292
I20260808 15:54:17.823961 17413 tablet_replica.cc:330] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: stopping tablet replica
I20260808 15:54:17.824362 17413 raft_consensus.cc:2229] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:17.824361 17035 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:17 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:44514
I20260808 15:54:17.824812 17413 raft_consensus.cc:2258] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:17.825078 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 16704
I20260808 15:54:17.830387 17413 ts_tablet_manager.cc:1905] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:17.835080 17413 ts_tablet_manager.cc:1918] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:17.835330 17413 log.cc:1196] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/a4e74beed6ec4d11a3d7183f3e8ac1a0
I20260808 15:54:17.835966 17413 ts_tablet_manager.cc:1939] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 234f42351148498e891e611b1922fc99: Deleting consensus metadata
I20260808 15:54:17.838061 16624 catalog_manager.cc:4841] TS 234f42351148498e891e611b1922fc99 (127.30.76.67:36837): tablet a4e74beed6ec4d11a3d7183f3e8ac1a0 (table test-table [id=72cdddabc4764071ac56aff3bc53a7da]) successfully deleted
I20260808 15:54:17.839774 17416 tablet_replica.cc:330] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: stopping tablet replica
I20260808 15:54:17.840163 17416 raft_consensus.cc:2229] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:54:17.840633 17416 raft_consensus.cc:2258] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:17.849040 17416 ts_tablet_manager.cc:1905] T a4e74beed6ec4d11a3d7183f3e8ac1a0 P 34f5c73c0d8747308871b4f8a88d224b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 15:54:17.849380 16622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:37181 (error 108)
I20260808 15:54:17.850332 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 16940
I20260808 15:54:17.851755 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.851374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:37181 (local address 127.0.0.1:43670)
0808 15:54:17.851589 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.851657 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:37181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260808 15:54:17.851912 16622 catalog_manager.cc:4584] TS 1f2e4c34a6ba43598fe42f7f2196ad18 (127.30.76.65:37181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4e74beed6ec4d11a3d7183f3e8ac1a0: Network error: Client connection negotiation failed: client connection to 127.30.76.65:37181: connect: Connection refused (error 111)
I20260808 15:54:17.861572 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 17138
I20260808 15:54:17.862296 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:17.861853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:32879 (local address 127.0.0.1:44540)
0808 15:54:17.862134 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:17.862196 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.66:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260808 15:54:17.872073 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 16601
I20260808 15:54:17.883802 31025 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 15:54:18.228444 31025 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 15:54:18.228554 31025 mini_ranger.cc:66] Stopping Ranger...
I20260808 15:54:18.583191 31025 mini_ranger.cc:68] Stopped Ranger
2026-08-08 15:54:18.583 UTC [15204] LOG: received smart shutdown request
2026-08-08 15:54:18.588 UTC [15204] LOG: background worker "logical replication launcher" (PID 15223) exited with exit code 1
2026-08-08 15:54:18.588 UTC [15214] LOG: shutting down
2026-08-08 15:54:18.746 UTC [15204] LOG: database system is shut down
2026-08-08T15:54:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50107 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:54:18 dist-test-slave-sb2z krb5kdc[17436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:54:18 dist-test-slave-sb2z krb5kdc[17436](info): set up 2 sockets
Aug 08 15:54:18 dist-test-slave-sb2z krb5kdc[17436](info): commencing operation
krb5kdc: starting...
W20260808 15:54:20.118945 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:54:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:54:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:54:20.242023 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:35151
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:35151,127.30.76.125:44699,127.30.76.124:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:20.351940 17457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:20.352201 17457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:20.352272 17457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:20.356140 17457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:20.356220 17457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:20.356259 17457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:20.356317 17457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:20.356357 17457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:20.361407 17457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:35151,127.30.76.125:44699,127.30.76.124:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:35151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:20.362380 17457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:20.363361 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:20.371647 17467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:20.371858 17457 server_base.cc:1034] running on GCE node
W20260808 15:54:20.371695 17470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:20.371665 17468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:20.372663 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:20.373685 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:20.374881 17457 hybrid_clock.cc:648] HybridClock initialized: now 1786204460374860 us; error 31 us; skew 500 ppm
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:20.380369 17457 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:20.381168 17457 webserver.cc:460] Webserver started at http://127.30.76.126:33553/ using document root <none> and password file <none>
I20260808 15:54:20.381659 17457 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:20.381717 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:20.381902 17457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:20.383852 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "7a4a9f4add204726a6fa96ed0d3f7891"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "64c7565a864c6a053fa30f0f2d92f030"
server_key_iv: "96ca3dfb688b35aed6016b8f2a6f4703"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.384358 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "7a4a9f4add204726a6fa96ed0d3f7891"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "64c7565a864c6a053fa30f0f2d92f030"
server_key_iv: "96ca3dfb688b35aed6016b8f2a6f4703"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.387998 17457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 15:54:20.390179 17480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:20.391301 17457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:20.391398 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "7a4a9f4add204726a6fa96ed0d3f7891"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "64c7565a864c6a053fa30f0f2d92f030"
server_key_iv: "96ca3dfb688b35aed6016b8f2a6f4703"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.391476 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:20.407876 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:20.411417 17457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:20.411654 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:20.419577 17457 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:35151
I20260808 15:54:20.419613 17550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:35151 every 8 connection(s)
I20260808 15:54:20.420511 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:20.423167 17552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:20.426836 17552 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:54:20.428031 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 17457
I20260808 15:54:20.428164 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:54:20.428421 31025 external_mini_cluster.cc:1428] Setting key 4eed7c70ac66402f1589252507b8da1a
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.125@KRBTEST.COM" created.
I20260808 15:54:20.443145 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.429198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57492 (local address 127.30.76.126:35151)
0808 15:54:20.429721 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.429740 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.430283 (+ 543us) server_negotiation.cc:407] Connection header received
0808 15:54:20.431734 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.431738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.431899 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.432128 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.432881 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.434399 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.435355 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.436033 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.436214 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.438470 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.438475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.438484 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.438506 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.440104 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.440537 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.440542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.440543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.440718 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.441075 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.441080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.441082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.441431 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.441533 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.442002 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.442244 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":310,"thread_start_us":138,"threads_started":1}
I20260808 15:54:20.443145 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.429198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:35151 (local address 127.0.0.1:57492)
0808 15:54:20.430126 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.430172 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.431417 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.432180 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.432185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.432496 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.432729 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.432742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.434526 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.434528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.435230 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.435240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.436201 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.436204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.436298 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.436879 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.436895 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.438333 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.440211 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.440219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.440226 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.440442 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.440832 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.440836 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.440838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.440973 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.441557 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.441566 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.441747 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.441941 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.442355 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":87,"spinlock_wait_cycles":14208,"thread_start_us":151,"threads_started":1}
I20260808 15:54:20.445695 17552 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:54:20.446444 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.446201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58548)
0808 15:54:20.446327 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.446391 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 15:54:20.446631 17483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
I20260808 15:54:20.447458 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.447255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58556)
0808 15:54:20.447361 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.447410 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:54:20.447642 17552 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:44699: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab.
Entry for principal kudu/127.30.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab.
I20260808 15:54:20.456729 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:44699
--webserver_interface=127.30.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab
--principal=kudu/127.30.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:35151,127.30.76.125:44699,127.30.76.124:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:20.496834 17552 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } attempt: 1
I20260808 15:54:20.497709 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.497410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58568)
0808 15:54:20.497570 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.497630 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 15:54:20.498426 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.498206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58572)
0808 15:54:20.498318 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.498372 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:54:20.498551 17552 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:44699: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
I20260808 15:54:20.566730 17552 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } attempt: 2
I20260808 15:54:20.568334 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.567351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58584)
0808 15:54:20.568172 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.568241 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260808 15:54:20.569155 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.568909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58586)
0808 15:54:20.569030 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.569084 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 15:54:20.569290 17552 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:44699: Network error: Client connection negotiation failed: client connection to 127.30.76.125:44699: connect: Connection refused (error 111)
W20260808 15:54:20.572746 17559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:20.572969 17559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:20.573064 17559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:20.577031 17559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:20.577080 17559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:20.577098 17559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:20.577116 17559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:20.577129 17559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:20.582247 17559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:35151,127.30.76.125:44699,127.30.76.124:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab
--principal=kudu/127.30.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:44699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:20.583250 17559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:20.584323 17559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:20.592496 17569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:20.592490 17566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:20.592764 17559 server_base.cc:1034] running on GCE node
W20260808 15:54:20.592500 17565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:20.593410 17559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:20.594403 17559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:20.595614 17559 hybrid_clock.cc:648] HybridClock initialized: now 1786204460595599 us; error 26 us; skew 500 ppm
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:20.598353 17559 init.cc:377] Logged in from keytab as kudu/127.30.76.125@KRBTEST.COM (short username kudu)
I20260808 15:54:20.599257 17559 webserver.cc:460] Webserver started at http://127.30.76.125:45857/ using document root <none> and password file <none>
I20260808 15:54:20.599854 17559 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:20.599917 17559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:20.600155 17559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:20.602073 17559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data/instance:
uuid: "ce56cc1028bc497b8805e0d8eb653524"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "dfffb059ac4477517d45ab7a651c460e"
server_key_iv: "77e1971c5652c5b1440582a2b5ec9d57"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.602624 17559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal/instance:
uuid: "ce56cc1028bc497b8805e0d8eb653524"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "dfffb059ac4477517d45ab7a651c460e"
server_key_iv: "77e1971c5652c5b1440582a2b5ec9d57"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.606216 17559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:54:20.608489 17582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:20.609623 17559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:20.609776 17559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal
uuid: "ce56cc1028bc497b8805e0d8eb653524"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "dfffb059ac4477517d45ab7a651c460e"
server_key_iv: "77e1971c5652c5b1440582a2b5ec9d57"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.609882 17559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:20.623397 17559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:20.625993 17559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:20.626185 17559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:20.632968 17559 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.125:44699
I20260808 15:54:20.633000 17654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.125:44699 every 8 connection(s)
I20260808 15:54:20.633857 17559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
I20260808 15:54:20.636423 17656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:20.640129 17656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:54:20.643422 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 17559
I20260808 15:54:20.643580 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-1/wal/instance
I20260808 15:54:20.643930 31025 external_mini_cluster.cc:1428] Setting key f5d59a73866e5d7b576f81504f366c24
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.124@KRBTEST.COM; defaulting to no policy
I20260808 15:54:20.654376 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.641902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.30.76.126:35151)
0808 15:54:20.642047 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.642051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.642812 (+ 761us) server_negotiation.cc:407] Connection header received
0808 15:54:20.643736 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.643739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.643805 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.643895 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.645014 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.645841 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.647620 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.648029 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.648150 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.650556 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.650563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.650568 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.650604 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.652347 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.652829 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.652835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.652837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.652930 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.653267 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.653270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.653272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.653494 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.653610 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.654084 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.654205 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:54:20.655092 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.642233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:35151 (local address 127.0.0.1:57506)
0808 15:54:20.642663 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.642706 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.643435 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.644080 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.644086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.644418 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.644865 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.644877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.645993 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.645999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.647450 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.647460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.648192 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.648195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.648331 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.648917 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.648933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.650388 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.652465 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.652469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.652480 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.652725 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.653032 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.653034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.653036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.653171 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.653617 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.653626 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.653855 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.654008 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.654253 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.124@KRBTEST.COM" created.
I20260808 15:54:20.656651 17656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:54:20.660502 17552 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } attempt: 3
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
I20260808 15:54:20.667385 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.657153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58592)
0808 15:54:20.657335 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.657351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.657425 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.657824 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.657829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.658032 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.658153 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.658162 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.659071 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.659077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.659644 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.659652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.660294 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.660298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.660391 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.661222 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.661235 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.662703 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.665579 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.665581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.665583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.665768 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.666070 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.666072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.666074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.666160 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.666713 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.666719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.666843 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.666965 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.667068 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 15:54:20.667434 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.657228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58592 (local address 127.30.76.125:44699)
0808 15:54:20.657517 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.657527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.657544 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:54:20.657577 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.657581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.657714 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.657813 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.658273 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.658960 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.659755 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.660181 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.660295 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.662866 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.662879 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.662894 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.662925 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.665483 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.665880 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.665885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.665886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.665986 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.666241 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.666243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.666244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.666567 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.666709 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.667073 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.667252 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":123,"threads_started":1}
I20260808 15:54:20.669746 17656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:54:20.670537 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.670311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46940)
0808 15:54:20.670417 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.670483 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:54:20.670794 17586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
I20260808 15:54:20.671186 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.660956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58608)
0808 15:54:20.661174 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.661192 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.661281 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.661542 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.661548 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.661739 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.661887 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.661899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.663055 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.663061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.663796 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.663806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.664272 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.664276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.664387 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.664901 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.664919 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.667387 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.669446 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.669453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.669458 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.669736 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.670050 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.670053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.670055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.670178 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.670630 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.670636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.670741 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.670860 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.670991 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 15:54:20.671257 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.661003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.30.76.125:44699)
0808 15:54:20.661303 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.661315 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.661330 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:54:20.661366 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.661369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.661411 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.661502 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.662026 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.662946 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.663915 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.664156 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.664267 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.667614 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.667620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.667622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.667660 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.669309 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.669834 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.669836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.669837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.669929 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.670292 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.670296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.670298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.670502 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.670614 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.670944 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.671064 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":107,"threads_started":1}
I20260808 15:54:20.671725 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.671479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46948)
0808 15:54:20.671606 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.671666 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 15:54:20.671885 17656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:39625: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab.
Entry for principal kudu/127.30.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab.
I20260808 15:54:20.672312 17552 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:54:20.673000 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.672782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46956)
0808 15:54:20.672897 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.672943 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 15:54:20.673161 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:39625
--webserver_interface=127.30.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab
--principal=kudu/127.30.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:35151,127.30.76.125:44699,127.30.76.124:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:20.673627 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.673400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46964)
0808 15:54:20.673535 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.673575 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 15:54:20.673830 17552 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:39625: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
I20260808 15:54:20.704975 17552 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } attempt: 1
I20260808 15:54:20.705781 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.705500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46972)
0808 15:54:20.705647 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.705705 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 15:54:20.706984 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.706757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46984)
0808 15:54:20.706879 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.706930 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 15:54:20.707118 17552 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:39625: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
I20260808 15:54:20.721133 17656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } attempt: 1
I20260808 15:54:20.721964 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.721684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:46992)
0808 15:54:20.721829 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.721887 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 15:54:20.722750 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.722527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47002)
0808 15:54:20.722647 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.722696 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:54:20.722883 17656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:39625: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
I20260808 15:54:20.782266 17552 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } attempt: 2
I20260808 15:54:20.783151 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.782840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47012)
0808 15:54:20.783000 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.783063 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 15:54:20.783910 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.783705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47022)
0808 15:54:20.783813 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.783861 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:54:20.784024 17552 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:39625: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
W20260808 15:54:20.787030 17667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:20.787303 17667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:20.787405 17667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 15:54:20.791114 17656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } attempt: 2
W20260808 15:54:20.791517 17667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:20.791599 17667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:20.791630 17667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:20.791653 17667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:20.791684 17667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:20.791873 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.791587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47032)
0808 15:54:20.791752 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.791809 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 15:54:20.792604 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.792412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47048)
0808 15:54:20.792510 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.792553 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 15:54:20.792800 17656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:39625: Network error: Client connection negotiation failed: client connection to 127.30.76.124:39625: connect: Connection refused (error 111)
I20260808 15:54:20.797220 17667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:35151,127.30.76.125:44699,127.30.76.124:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab
--principal=kudu/127.30.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:20.798257 17667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:20.799239 17667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:20.807492 17676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:20.807740 17674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:20.807773 17673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:20.808077 17667 server_base.cc:1034] running on GCE node
I20260808 15:54:20.808645 17667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:20.809739 17667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:20.810936 17667 hybrid_clock.cc:648] HybridClock initialized: now 1786204460810918 us; error 31 us; skew 500 ppm
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:20.813871 17667 init.cc:377] Logged in from keytab as kudu/127.30.76.124@KRBTEST.COM (short username kudu)
I20260808 15:54:20.814747 17667 webserver.cc:460] Webserver started at http://127.30.76.124:41417/ using document root <none> and password file <none>
I20260808 15:54:20.815310 17667 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:20.815397 17667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:20.815683 17667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:20.817739 17667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data/instance:
uuid: "d4b048efd666485c8c07f39b27ba6638"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "c09398e65b8a724d92d77257079c2024"
server_key_iv: "5837de876446352995d18c66b9da4664"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.818323 17667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal/instance:
uuid: "d4b048efd666485c8c07f39b27ba6638"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "c09398e65b8a724d92d77257079c2024"
server_key_iv: "5837de876446352995d18c66b9da4664"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.821934 17667 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 15:54:20.824223 17685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:20.825381 17667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:54:20.825529 17667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal
uuid: "d4b048efd666485c8c07f39b27ba6638"
format_stamp: "Formatted at 2026-08-08 15:54:20 on dist-test-slave-sb2z"
server_key: "c09398e65b8a724d92d77257079c2024"
server_key_iv: "5837de876446352995d18c66b9da4664"
server_key_version: "encryptionkey@0"
I20260808 15:54:20.825636 17667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:20.836233 17667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:20.839286 17667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:20.839473 17667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:20.846243 17667 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.124:39625
I20260808 15:54:20.846302 17763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.124:39625 every 8 connection(s)
I20260808 15:54:20.847214 17667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
I20260808 15:54:20.849857 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 17667
I20260808 15:54:20.849987 17765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:20.849999 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786204294980517-31025-0/minicluster-data/master-2/wal/instance
I20260808 15:54:20.850309 31025 external_mini_cluster.cc:1428] Setting key eab9b2cc71a05867b8fd587d2db60a0e
I20260808 15:54:20.855155 17765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:20.864872 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.851620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.30.76.124:39625)
0808 15:54:20.852017 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.852035 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.852078 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:54:20.852867 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.852891 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.853303 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.853666 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.854308 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.855748 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.856569 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.857211 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.857375 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.859871 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.859877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.859893 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.859918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.861787 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.862242 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.862248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.862253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.862379 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.862685 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.862687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.862688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.863178 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.863342 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.863721 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.864012 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260808 15:54:20.866581 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.856838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:35151 (local address 127.0.0.1:57508)
0808 15:54:20.857228 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.857272 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.857425 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.857746 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.857752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.858098 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.858221 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.858226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.858944 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.858947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.859747 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.859757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.860223 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.860226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.860354 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.860789 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.860803 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.862465 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.864333 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.864340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.864345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.864614 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.864888 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.864894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.864896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.864989 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.866110 (+ 1121us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.866114 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.866204 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.866312 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.866432 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":127,"threads_started":1}
I20260808 15:54:20.866849 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.856930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.30.76.126:35151)
0808 15:54:20.857077 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.857083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.857345 (+ 262us) server_negotiation.cc:407] Connection header received
0808 15:54:20.857550 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.857558 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.857603 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.857699 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.858344 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.858822 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.859864 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.860079 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.860231 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.862598 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.862603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.862608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.862641 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.864229 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.864721 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.864723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.864724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.864795 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.865788 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.865792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.865793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.866005 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.866109 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.866411 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.866539 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:54:20.867803 17765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
I20260808 15:54:20.876039 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.867210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.30.76.126:35151)
0808 15:54:20.867363 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.867367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.867380 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:54:20.867417 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.867419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.867458 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.867589 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.868217 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.868887 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.869953 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.870273 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.870371 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.872826 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.872832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.872835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.872866 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.874288 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.874728 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.874731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.874733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.874813 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.875121 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.875124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.875127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.875339 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.875588 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.875781 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.875877 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":100,"threads_started":1}
I20260808 15:54:20.876529 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.868243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:44699 (local address 127.0.0.1:58614)
0808 15:54:20.868376 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.868388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.868451 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.868703 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.868706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.868858 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.868952 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.868962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.869953 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.869956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.870636 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.870644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.871107 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.871111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.871196 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.871664 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.871675 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.873121 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.874905 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.874911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.874913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.875147 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.875415 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.875421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.875426 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.875519 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.876093 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.876096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.876204 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.876311 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.876436 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 15:54:20.876665 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.868368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.30.76.125:44699)
0808 15:54:20.868482 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.868486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.868505 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:54:20.868543 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.868545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.868581 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.868667 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.869079 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.869846 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.870758 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.870994 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.871106 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.873302 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.873306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.873308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.873332 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.874800 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.875245 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.875248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.875249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.875320 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.875634 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.875638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.875640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.875878 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.875977 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.876382 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.876500 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:54:20.877326 17765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:54:20.883183 17552 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } attempt: 3
I20260808 15:54:20.884060 17656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } attempt: 3
I20260808 15:54:20.886849 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.877149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.30.76.125:44699)
0808 15:54:20.877237 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.877243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.877271 (+ 28us) server_negotiation.cc:407] Connection header received
0808 15:54:20.877534 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.877554 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.877602 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.877685 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.878403 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.879119 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.880033 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.880232 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.880340 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.882588 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.882591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.882593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.882613 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.884641 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.885127 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.885130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.885132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.885237 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.885565 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.885570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.885572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.885793 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.885998 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.886590 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.886706 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 15:54:20.886965 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.877867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.30.76.124:39625)
0808 15:54:20.877966 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.877975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.878050 (+ 75us) server_negotiation.cc:407] Connection header received
0808 15:54:20.878174 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.878176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.878205 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.878271 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.878704 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.879401 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.880338 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.880602 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.880682 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.883108 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.883115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.883118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.883143 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.885438 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.885891 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.885894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.885895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.885963 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.886184 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.886186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.886188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.886378 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.886439 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.886728 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.886829 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:54:20.887458 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.877821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47080)
0808 15:54:20.877989 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.878005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.878095 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.878314 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.878318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.878495 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.878579 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.878584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.879502 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.879505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.880221 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.880230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.880716 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.880722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.880821 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.881245 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.881260 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.882979 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.885543 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.885546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.885548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.885789 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.886046 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.886049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.886051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.886112 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.886495 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.886497 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.886568 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.887175 (+ 607us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.887308 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:54:20.890910 17765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638: Bootstrap starting.
I20260808 15:54:20.890954 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:54:20.893918 17765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:20.894668 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.884438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47086)
0808 15:54:20.884606 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.884619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.884714 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.885921 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.885923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.886088 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.886206 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.886217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.887349 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.887353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.888564 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.888575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.889177 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.889180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.889258 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.889830 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.889844 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.891180 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.892969 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.892975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.892979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.893186 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.893501 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.893504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.893506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.893596 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.894142 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.894148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.894265 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.894346 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.894481 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 15:54:20.894821 17765 log.cc:824] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:20.895350 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.884470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.30.76.124:39625)
0808 15:54:20.885704 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.885713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.885725 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:54:20.885755 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.885757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.885795 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.886318 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.886450 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.887217 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.888791 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.889053 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.889175 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.891309 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.891312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.891314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.891336 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.892860 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.893286 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.893292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.893294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.893386 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.893732 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.893734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.893738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.893951 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.894950 (+ 999us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.895022 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.895154 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170,"thread_start_us":88,"threads_started":1}
Aug 08 15:54:20 dist-test-slave-sb2z krb5kdc[17436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204460, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:54:20.897285 17765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638: No bootstrap required, opened a new log
I20260808 15:54:20.898701 17656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: Bootstrap starting.
I20260808 15:54:20.900345 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.883675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:39625 (local address 127.0.0.1:47082)
0808 15:54:20.883851 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:20.883866 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:20.883963 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:20.884496 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:20.884499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:20.884668 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:20.884775 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.884781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.886323 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.886326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.887097 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:20.887111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.887851 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.887854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:20.887939 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.895294 (+ 7355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:20.895309 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:20.896765 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:20.898729 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.898732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.898734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.898971 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.899267 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:20.899270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:20.899272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:20.899361 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:20.899860 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:20.899863 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:20.899959 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 15:54:20.900056 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 15:54:20.900185 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 15:54:20.900902 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.883664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.30.76.124:39625)
0808 15:54:20.883964 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.883973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.883989 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:54:20.884054 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.884056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.884099 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.884872 (+ 773us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.884940 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.886202 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.887398 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.887731 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.887847 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.896917 (+ 9070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.896921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.896923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.896948 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.898590 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.899082 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.899085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.899087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.899162 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.899462 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.899467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.899470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.899722 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.899814 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.900589 (+ 775us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.900720 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":115,"threads_started":1}
I20260808 15:54:20.901392 17656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:20.902274 17656 log.cc:824] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:20.902294 17765 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.902516 17765 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:20.902567 17765 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b048efd666485c8c07f39b27ba6638, State: Initialized, Role: FOLLOWER
I20260808 15:54:20.903020 17765 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.903913 17552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: Bootstrap starting.
I20260808 15:54:20.904408 17765 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:20.905318 17656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: No bootstrap required, opened a new log
I20260808 15:54:20.906559 17552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:20.907168 17552 log.cc:824] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:20.908100 17795 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.908241 17795 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:20.910547 17656 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.910853 17656 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:20.910962 17656 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce56cc1028bc497b8805e0d8eb653524, State: Initialized, Role: FOLLOWER
I20260808 15:54:20.911562 17656 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
W20260808 15:54:20.911674 17809 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 15:54:20.908806 17667 master_runner.cc:186] Error getting master registration for 127.30.76.126:35151: OK, instance_id { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" instance_seqno: 1786204460417330 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:54:20.911926 17667 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 15:54:20.912050 17809 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:54:20.912243 17815 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.912576 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.893237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.30.76.126:35151)
0808 15:54:20.893342 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.893345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.894205 (+ 860us) server_negotiation.cc:407] Connection header received
0808 15:54:20.894890 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.894894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.894943 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.895013 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.896880 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.898104 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.900963 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.901300 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.901384 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.907778 (+ 6394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.907783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.907785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.907811 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.909479 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.910404 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.910407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.910409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.910495 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.911269 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.911273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.911275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.911483 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.911642 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.912280 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.912404 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:54:20.912585 17815 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:20.912875 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.893510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.30.76.124:39625)
0808 15:54:20.893608 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.893612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.894416 (+ 804us) server_negotiation.cc:407] Connection header received
0808 15:54:20.894663 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.894666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.894710 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.894759 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.899195 (+ 4436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.899876 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.902005 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.902333 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.902415 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.905290 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.905294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.905296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.905320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.906984 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.909973 (+ 2989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.909977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.909979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.910066 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.910824 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.910828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.910829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.911061 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.911184 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.912639 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.912751 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:54:20.913128 17656 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:20.916173 17552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: No bootstrap required, opened a new log
I20260808 15:54:20.920425 17552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.920814 17552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:20.920939 17552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4a9f4add204726a6fa96ed0d3f7891, State: Initialized, Role: FOLLOWER
I20260808 15:54:20.917484 17559 master_runner.cc:186] Error getting master registration for 127.30.76.126:35151: OK, instance_id { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" instance_seqno: 1786204460417330 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:54:20.921506 17559 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:54:20.921504 17552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.922298 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:20.893917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58646 (local address 127.30.76.125:44699)
0808 15:54:20.894018 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:20.894021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:20.912511 (+ 18490us) server_negotiation.cc:407] Connection header received
0808 15:54:20.913502 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:20.913505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:20.913563 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:20.913624 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:20.914523 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.915171 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.916296 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:20.916641 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:20.916742 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:20.918388 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:20.918391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:20.918394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:20.918419 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:20.920100 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.920601 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.920605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.920607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.920693 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:20.921024 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:20.921027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:20.921029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:20.921246 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:20.921365 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:20.922034 (+ 669us) server_negotiation.cc:299] Negotiation successful
0808 15:54:20.922160 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:54:20.923681 17552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 15:54:20.924520 17836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:54:20.927553 17836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 15:54:20.931250 17858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:54:20.931372 17858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:54:20.934538 17457 master_runner.cc:263] No leader master found from master 7a4a9f4add204726a6fa96ed0d3f7891
I20260808 15:54:20.934603 17457 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:54:20.935988 17837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.936115 17837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:20.948889 17795 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:20.949043 17795 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.949965 17795 leader_election.cc:288] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a4a9f4add204726a6fa96ed0d3f7891 (127.30.76.126:35151), ce56cc1028bc497b8805e0d8eb653524 (127.30.76.125:44699)
I20260808 15:54:20.950977 17618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4b048efd666485c8c07f39b27ba6638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce56cc1028bc497b8805e0d8eb653524" is_pre_election: true
I20260808 15:54:20.951269 17618 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4b048efd666485c8c07f39b27ba6638 in term 0.
I20260808 15:54:20.951678 17517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4b048efd666485c8c07f39b27ba6638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" is_pre_election: true
I20260808 15:54:20.952490 17517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4b048efd666485c8c07f39b27ba6638 in term 0.
I20260808 15:54:20.952943 17688 leader_election.cc:302] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce56cc1028bc497b8805e0d8eb653524, d4b048efd666485c8c07f39b27ba6638; no voters:
I20260808 15:54:20.953297 17795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:54:20.953378 17795 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:54:20.953423 17795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:20.954737 17795 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.955087 17795 leader_election.cc:288] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [CANDIDATE]: Term 1 election: Requested vote from peers 7a4a9f4add204726a6fa96ed0d3f7891 (127.30.76.126:35151), ce56cc1028bc497b8805e0d8eb653524 (127.30.76.125:44699)
I20260808 15:54:20.955416 17517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4b048efd666485c8c07f39b27ba6638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a4a9f4add204726a6fa96ed0d3f7891"
I20260808 15:54:20.955578 17517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:20.956863 17517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b048efd666485c8c07f39b27ba6638 in term 1.
I20260808 15:54:20.957263 17691 leader_election.cc:302] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a4a9f4add204726a6fa96ed0d3f7891, d4b048efd666485c8c07f39b27ba6638; no voters:
I20260808 15:54:20.957513 17618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4b048efd666485c8c07f39b27ba6638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce56cc1028bc497b8805e0d8eb653524"
I20260808 15:54:20.957628 17618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:20.959296 17618 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b048efd666485c8c07f39b27ba6638 in term 1.
I20260808 15:54:20.959822 17795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:20.960039 17795 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [term 1 LEADER]: Becoming Leader. State: Replica: d4b048efd666485c8c07f39b27ba6638, State: Running, Role: LEADER
I20260808 15:54:20.960402 17795 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } }
I20260808 15:54:20.965375 17795 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b048efd666485c8c07f39b27ba6638. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.965538 17795 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:20.965945 17866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:20.966928 17866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:20.974201 17618 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [term 1 FOLLOWER]: Refusing update from remote peer d4b048efd666485c8c07f39b27ba6638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:20.977027 17861 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:20.981530 17815 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b048efd666485c8c07f39b27ba6638. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.981662 17815 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:20.982688 17517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [term 1 FOLLOWER]: Refusing update from remote peer d4b048efd666485c8c07f39b27ba6638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:20.983265 17861 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:20.987196 17837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b048efd666485c8c07f39b27ba6638. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.987313 17837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:20.993320 17861 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.993450 17861 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:20.994483 17837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.994592 17837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:20.994814 17867 mvcc.cc:205] Tried to move back new op lower bound from 7316293472142823424 to 7316293472106049536. Current Snapshot: MvccSnapshot[applied={T|T < 7316293472142823424}]
I20260808 15:54:20.996609 17870 mvcc.cc:205] Tried to move back new op lower bound from 7316293472142823424 to 7316293472106049536. Current Snapshot: MvccSnapshot[applied={T|T < 7316293472142823424}]
I20260808 15:54:20.999337 17795 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:20.999440 17795 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:21.000546 17815 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4b048efd666485c8c07f39b27ba6638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4a9f4add204726a6fa96ed0d3f7891" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 35151 } } peers { permanent_uuid: "ce56cc1028bc497b8805e0d8eb653524" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 44699 } } peers { permanent_uuid: "d4b048efd666485c8c07f39b27ba6638" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 39625 } } }
I20260808 15:54:21.000658 17815 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:54:21.001511 17868 mvcc.cc:205] Tried to move back new op lower bound from 7316293472142823424 to 7316293472106049536. Current Snapshot: MvccSnapshot[applied={T|T < 7316293472142823424}]
I20260808 15:54:21.007005 17866 catalog_manager.cc:1347] Generated new cluster ID: 5bbc6e50c8494d3db1d9be197d4814a4
I20260808 15:54:21.007071 17866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:21.039801 17866 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:54:21.041198 17866 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:21.065181 17866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4b048efd666485c8c07f39b27ba6638: Generated new TSK 0
I20260808 15:54:21.066111 17866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:54:21.929023 17836 catalog_manager.cc:1259] Loaded cluster ID: 5bbc6e50c8494d3db1d9be197d4814a4
I20260808 15:54:21.929108 17836 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: loading cluster ID for follower catalog manager: success
I20260808 15:54:21.930822 17836 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: acquiring CA information for follower catalog manager: success
I20260808 15:54:21.932013 17836 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce56cc1028bc497b8805e0d8eb653524: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:54:21.932513 17858 catalog_manager.cc:1259] Loaded cluster ID: 5bbc6e50c8494d3db1d9be197d4814a4
I20260808 15:54:21.932572 17858 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: loading cluster ID for follower catalog manager: success
I20260808 15:54:21.933992 17858 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: acquiring CA information for follower catalog manager: success
I20260808 15:54:21.934962 17858 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a4a9f4add204726a6fa96ed0d3f7891: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:54:22.141093 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 17457
I20260808 15:54:22.153755 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 17559
I20260808 15:54:22.165714 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 17667
2026-08-08T15:54:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3278 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:54:22 dist-test-slave-sb2z krb5kdc[17889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:54:22 dist-test-slave-sb2z krb5kdc[17889](info): set up 2 sockets
Aug 08 15:54:22 dist-test-slave-sb2z krb5kdc[17889](info): commencing operation
krb5kdc: starting...
W20260808 15:54:23.369240 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:54:23 dist-test-slave-sb2z krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:54:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:54:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:54:23.491447 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:46511
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:46511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:23.605453 17908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:23.605690 17908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:23.605729 17908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:23.609632 17908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:23.609685 17908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:23.609704 17908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:23.609720 17908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:23.609783 17908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:23.614774 17908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:46511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:23.615785 17908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:23.616778 17908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:23.625221 17914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:23.625236 17916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:23.625377 17908 server_base.cc:1034] running on GCE node
W20260808 15:54:23.625186 17913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:23.626153 17908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:23.627185 17908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:23.628399 17908 hybrid_clock.cc:648] HybridClock initialized: now 1786204463628376 us; error 42 us; skew 500 ppm
Aug 08 15:54:23 dist-test-slave-sb2z krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204463, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:23.631147 17908 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:23.632081 17908 webserver.cc:460] Webserver started at http://127.30.76.126:42697/ using document root <none> and password file <none>
I20260808 15:54:23.632670 17908 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:23.632761 17908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:23.632989 17908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:23.635051 17908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "da702913391943fa8972f8e7ada8c0ae"
format_stamp: "Formatted at 2026-08-08 15:54:23 on dist-test-slave-sb2z"
server_key: "9218a53d630fb766894ead550143069e"
server_key_iv: "77e97ea38110340a24e92d49395b0847"
server_key_version: "encryptionkey@0"
I20260808 15:54:23.635682 17908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "da702913391943fa8972f8e7ada8c0ae"
format_stamp: "Formatted at 2026-08-08 15:54:23 on dist-test-slave-sb2z"
server_key: "9218a53d630fb766894ead550143069e"
server_key_iv: "77e97ea38110340a24e92d49395b0847"
server_key_version: "encryptionkey@0"
I20260808 15:54:23.639210 17908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:54:23.641559 17929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:23.642745 17908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:23.642897 17908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "da702913391943fa8972f8e7ada8c0ae"
format_stamp: "Formatted at 2026-08-08 15:54:23 on dist-test-slave-sb2z"
server_key: "9218a53d630fb766894ead550143069e"
server_key_iv: "77e97ea38110340a24e92d49395b0847"
server_key_version: "encryptionkey@0"
I20260808 15:54:23.643007 17908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:23.665771 17908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:23.668602 17908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:23.668936 17908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:23.676764 17908 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:46511
I20260808 15:54:23.676771 17997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:46511 every 8 connection(s)
I20260808 15:54:23.677799 17908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:23.680548 17998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:23.684719 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 17908
I20260808 15:54:23.684875 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:54:23.685155 31025 external_mini_cluster.cc:1428] Setting key b8328f1749259d4ca364877f2b692cb4
I20260808 15:54:23.686272 17998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae: Bootstrap starting.
I20260808 15:54:23.688540 17998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:23.689443 17998 log.cc:824] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:23.691512 17998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae: No bootstrap required, opened a new log
Aug 08 15:54:23 dist-test-slave-sb2z krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:23.695493 17998 raft_consensus.cc:348] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da702913391943fa8972f8e7ada8c0ae" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46511 } }
I20260808 15:54:23.695801 17998 raft_consensus.cc:374] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:23.695868 17998 raft_consensus.cc:729] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da702913391943fa8972f8e7ada8c0ae, State: Initialized, Role: FOLLOWER
I20260808 15:54:23.696336 17998 consensus_queue.cc:260] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da702913391943fa8972f8e7ada8c0ae" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46511 } }
I20260808 15:54:23.696509 17998 raft_consensus.cc:388] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:23.696585 17998 raft_consensus.cc:482] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:23.696712 17998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:23.698225 17998 raft_consensus.cc:504] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da702913391943fa8972f8e7ada8c0ae" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46511 } }
I20260808 15:54:23.698587 17998 leader_election.cc:302] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da702913391943fa8972f8e7ada8c0ae; no voters:
I20260808 15:54:23.698951 17998 leader_election.cc:288] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:54:23.700490 17998 sys_catalog.cc:564] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:23.700829 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:23.686505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.30.76.126:46511)
0808 15:54:23.686922 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:23.686937 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:23.686972 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:54:23.687674 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:23.687693 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:23.688006 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:23.688276 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:23.689229 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:23.690282 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:23.691110 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:23.691833 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:23.691997 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:23.694386 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:23.694392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:23.694407 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:23.694438 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:23.696412 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:23.697049 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:23.697055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:23.697063 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:23.697235 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:23.697596 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:23.697599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:23.697600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:23.698264 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:23.698660 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:23.699091 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 15:54:23.699495 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":144,"threads_started":1}
I20260808 15:54:23.703115 18009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:23.703332 18009 raft_consensus.cc:686] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [term 1 LEADER]: Becoming Leader. State: Replica: da702913391943fa8972f8e7ada8c0ae, State: Running, Role: LEADER
I20260808 15:54:23.703684 18009 consensus_queue.cc:237] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da702913391943fa8972f8e7ada8c0ae" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46511 } }
I20260808 15:54:23.705013 18030 sys_catalog.cc:455] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da702913391943fa8972f8e7ada8c0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da702913391943fa8972f8e7ada8c0ae" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46511 } } }
I20260808 15:54:23.705171 18030 sys_catalog.cc:458] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:23.705025 18009 sys_catalog.cc:455] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader da702913391943fa8972f8e7ada8c0ae. Latest consensus state: current_term: 1 leader_uuid: "da702913391943fa8972f8e7ada8c0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da702913391943fa8972f8e7ada8c0ae" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46511 } } }
I20260808 15:54:23.705376 18009 sys_catalog.cc:458] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae [sys.catalog]: This master's current role is: LEADER
W20260808 15:54:23.706724 18028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:54:23.706845 18028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:54:23.707005 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:54:23.707980 18035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:23.708972 18035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:23.714468 18035 catalog_manager.cc:1347] Generated new cluster ID: 903a1364bed44b92b1acaa54853d87ce
I20260808 15:54:23.714533 18035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:23.736936 18035 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:54:23.737864 18035 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:23.742132 18035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da702913391943fa8972f8e7ada8c0ae: Generated new TSK 0
I20260808 15:54:23.742763 18035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:54:23.763304 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 17908
2026-08-08T15:54:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1600 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:54:23 dist-test-slave-sb2z krb5kdc[18043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:54:23 dist-test-slave-sb2z krb5kdc[18043](info): set up 2 sockets
Aug 08 15:54:23 dist-test-slave-sb2z krb5kdc[18043](info): commencing operation
krb5kdc: starting...
W20260808 15:54:24.989071 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:54:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:54:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:54:25.114574 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:43607
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:25.229460 18062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:25.229868 18062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:25.229943 18062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:25.233958 18062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:25.234052 18062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:25.234092 18062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:25.234130 18062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:25.234164 18062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:25.240824 18062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:25.241925 18062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:25.242986 18062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:25.251701 18073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.251706 18075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:25.251940 18062 server_base.cc:1034] running on GCE node
W20260808 15:54:25.251760 18071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:25.252660 18062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:25.253729 18062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:25.254914 18062 hybrid_clock.cc:648] HybridClock initialized: now 1786204465254883 us; error 44 us; skew 500 ppm
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:25.257606 18062 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:25.258453 18062 webserver.cc:460] Webserver started at http://127.30.76.126:34363/ using document root <none> and password file <none>
I20260808 15:54:25.259001 18062 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:25.259091 18062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:25.259335 18062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:25.261443 18062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "81cdafedf1024c639557bd6e95fe074b"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "05671ca2f7fd17b5a610e76eabc81409"
server_key_iv: "18a4661b34dc0f647c1e39ae6671ad5e"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.262002 18062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "81cdafedf1024c639557bd6e95fe074b"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "05671ca2f7fd17b5a610e76eabc81409"
server_key_iv: "18a4661b34dc0f647c1e39ae6671ad5e"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.265583 18062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:54:25.267889 18089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.270198 18062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:54:25.270455 18062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "81cdafedf1024c639557bd6e95fe074b"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "05671ca2f7fd17b5a610e76eabc81409"
server_key_iv: "18a4661b34dc0f647c1e39ae6671ad5e"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.270684 18062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:25.297022 18062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:25.299820 18062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:25.299969 18062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:25.307189 18062 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:43607
I20260808 15:54:25.307510 18168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:43607 every 8 connection(s)
I20260808 15:54:25.308209 18062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:25.311056 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 18062
I20260808 15:54:25.311115 18169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:25.311223 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:54:25.311586 31025 external_mini_cluster.cc:1428] Setting key 2f4d3688ddd73d9f8c3acd4481e23e23
I20260808 15:54:25.317929 18169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap starting.
I20260808 15:54:25.320464 18169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:25.321427 18169 log.cc:824] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:25.324079 18169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: No bootstrap required, opened a new log
I20260808 15:54:25.328890 18169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:25.329142 18169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:25.329176 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:25.312926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.30.76.126:43607)
0808 15:54:25.313328 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:25.313343 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:25.313378 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:54:25.314258 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:25.314277 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:25.314579 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:25.314843 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:25.315755 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.316897 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.318073 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.319330 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.319639 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:25.322718 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:25.322724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:25.322740 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:25.322769 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:25.325623 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:25.326196 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:25.326203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:25.326209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:25.326368 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:25.326704 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:25.326707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:25.326708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:25.327334 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:25.327567 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:25.327948 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 15:54:25.328254 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":128,"threads_started":1}
I20260808 15:54:25.329221 18169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cdafedf1024c639557bd6e95fe074b, State: Initialized, Role: FOLLOWER
I20260808 15:54:25.329684 18169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:25.329833 18169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:25.329893 18169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:25.329981 18169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:25.330912 18169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:25.331277 18169 leader_election.cc:302] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cdafedf1024c639557bd6e95fe074b; no voters:
I20260808 15:54:25.331691 18169 leader_election.cc:288] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:54:25.331771 18175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:25.332036 18175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 LEADER]: Becoming Leader. State: Replica: 81cdafedf1024c639557bd6e95fe074b, State: Running, Role: LEADER
I20260808 15:54:25.332329 18175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:25.332872 18169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:25.333817 18179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cdafedf1024c639557bd6e95fe074b. Latest consensus state: current_term: 1 leader_uuid: "81cdafedf1024c639557bd6e95fe074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } } }
I20260808 15:54:25.333951 18179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:25.334393 18177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81cdafedf1024c639557bd6e95fe074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } } }
I20260808 15:54:25.334488 18177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:25.335081 18187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:25.337618 18187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:25.343024 18187 catalog_manager.cc:1347] Generated new cluster ID: c89494ae0f564f01951ea0d6f14c6136
I20260808 15:54:25.343089 18187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:25.352658 18187 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:54:25.353508 18187 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:25.360157 18187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Generated new TSK 0
I20260808 15:54:25.360760 18187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:54:25.393818 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:43607
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:25.506142 18203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:25.506419 18203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:25.506462 18203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:25.510450 18203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:25.510504 18203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:25.510588 18203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:54:25.515697 18203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:25.516744 18203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:25.517781 18203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:25.527908 18212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.527942 18216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.527925 18214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:25.528343 18203 server_base.cc:1034] running on GCE node
I20260808 15:54:25.528884 18203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:25.529717 18203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:25.530917 18203 hybrid_clock.cc:648] HybridClock initialized: now 1786204465530888 us; error 49 us; skew 500 ppm
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:25.538933 18203 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:54:25.539948 18203 webserver.cc:460] Webserver started at http://127.30.76.65:45099/ using document root <none> and password file <none>
I20260808 15:54:25.540540 18203 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:25.540637 18203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:25.540882 18203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:25.542961 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "629015e49462454988aa207e71601766"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "14e8e0ac1575f9dc58e4e630143f97fc"
server_key_iv: "bd8c5407a2387751f6cafde06eb343da"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.543574 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "629015e49462454988aa207e71601766"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "14e8e0ac1575f9dc58e4e630143f97fc"
server_key_iv: "bd8c5407a2387751f6cafde06eb343da"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.547335 18203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:54:25.549716 18230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.550840 18203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:25.550976 18203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "629015e49462454988aa207e71601766"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "14e8e0ac1575f9dc58e4e630143f97fc"
server_key_iv: "bd8c5407a2387751f6cafde06eb343da"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.551086 18203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:25.574617 18203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:25.577885 18203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:25.578089 18203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:25.578681 18203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:25.579507 18203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:25.579627 18203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.579702 18203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:25.579743 18203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.588466 18203 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:42031
I20260808 15:54:25.588495 18383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:42031 every 8 connection(s)
I20260808 15:54:25.589335 18203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:54:25.591382 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 18203
I20260808 15:54:25.591569 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:54:25.591935 31025 external_mini_cluster.cc:1428] Setting key 3ec2ca863f5fd3f672cecc1a3e15bdd6
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:54:25.604640 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:25.591108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:60903 (local address 127.30.76.126:43607)
0808 15:54:25.592417 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:25.592422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:25.592440 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:54:25.593914 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:25.593916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:25.593960 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:25.594026 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:25.595615 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.596168 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.597714 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.597932 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.598048 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:25.600893 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:25.600896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:25.600898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:25.600931 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:25.602356 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:25.602957 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:25.602960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:25.602961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:25.603032 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:25.603498 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:25.603500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:25.603501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:25.603721 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:25.603797 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:25.604422 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 15:54:25.604519 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217}
I20260808 15:54:25.605505 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:25.591450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:60903)
0808 15:54:25.592082 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:25.592142 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:25.593693 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:25.594201 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:25.594212 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:25.594748 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:25.595423 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:25.595444 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.596300 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.596306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:25.597604 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:25.597615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.598093 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.598097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:25.598265 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:25.598856 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:25.598872 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:25.600722 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:25.602477 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:25.602483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:25.602496 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:25.602856 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:25.603147 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:25.603153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:25.603156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:25.603391 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:25.603827 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:25.603834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:25.604156 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 15:54:25.604366 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 15:54:25.604617 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:54:25.606801 18385 heartbeater.cc:347] Connected to a master server at 127.30.76.126:43607
I20260808 15:54:25.607064 18385 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:25.607679 18385 heartbeater.cc:511] Master 127.30.76.126:43607 requested a full tablet report, sending...
I20260808 15:54:25.609516 18108 ts_manager.cc:194] Registered new tserver with Master: 629015e49462454988aa207e71601766 (127.30.76.65:42031)
I20260808 15:54:25.611241 18108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:60903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:54:25.622079 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:43607
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:25.730911 18392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:25.731161 18392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:25.731204 18392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:25.735244 18392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:25.735299 18392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:25.735356 18392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:54:25.740530 18392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:25.741506 18392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:25.742457 18392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:25.751966 18405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.752162 18406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.752199 18409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:25.752213 18392 server_base.cc:1034] running on GCE node
I20260808 15:54:25.752717 18392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:25.753360 18392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:25.754524 18392 hybrid_clock.cc:648] HybridClock initialized: now 1786204465754540 us; error 79 us; skew 500 ppm
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:25.757347 18392 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:54:25.758217 18392 webserver.cc:460] Webserver started at http://127.30.76.66:36407/ using document root <none> and password file <none>
I20260808 15:54:25.758718 18392 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:25.758777 18392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:25.758937 18392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:25.760943 18392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "8d30331485054a299246450a432e7c1e"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "7d1ea9430f53507851f8cd987b164595"
server_key_iv: "ea35b71afca932c90df309f810d8a86b"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.761469 18392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "8d30331485054a299246450a432e7c1e"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "7d1ea9430f53507851f8cd987b164595"
server_key_iv: "ea35b71afca932c90df309f810d8a86b"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.765018 18392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:54:25.767288 18421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.768621 18392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:25.768757 18392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "8d30331485054a299246450a432e7c1e"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "7d1ea9430f53507851f8cd987b164595"
server_key_iv: "ea35b71afca932c90df309f810d8a86b"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.768868 18392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:25.785410 18392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:25.788607 18392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:25.788796 18392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:25.789371 18392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:25.790153 18392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:25.790225 18392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.790304 18392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:25.790350 18392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.798853 18392 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:38843
I20260808 15:54:25.798873 18582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:38843 every 8 connection(s)
I20260808 15:54:25.799697 18392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:54:25.808481 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 18392
I20260808 15:54:25.808621 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:25.808964 31025 external_mini_cluster.cc:1428] Setting key 5734836925797a527bd2e7b2513c6fbf
I20260808 15:54:25.813359 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:25.801440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:40095 (local address 127.30.76.126:43607)
0808 15:54:25.801577 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:25.801580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:25.802285 (+ 705us) server_negotiation.cc:407] Connection header received
0808 15:54:25.803675 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:25.803678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:25.803744 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:25.803840 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:25.804923 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.805445 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.806574 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.806824 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.806939 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:25.809598 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:25.809606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:25.809608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:25.809637 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:25.811254 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:25.811814 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:25.811820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:25.811825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:25.811920 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:25.812248 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:25.812255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:25.812259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:25.812504 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:25.812606 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:25.813064 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 15:54:25.813191 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:54:25.814073 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:25.801710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:40095)
0808 15:54:25.802128 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:25.802177 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:25.803412 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:25.804006 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:25.804012 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:25.804338 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:25.804770 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:25.804783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.805574 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.805578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:25.806439 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:25.806448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:25.806991 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:25.806996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:25.807171 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:25.807735 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:25.807751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:25.809429 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:25.811374 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:25.811378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:25.811389 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:25.811715 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:25.812015 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:25.812017 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:25.812018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:25.812146 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:25.812604 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:25.812608 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:25.812820 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 15:54:25.813001 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 15:54:25.813299 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":114,"threads_started":1}
I20260808 15:54:25.815107 18583 heartbeater.cc:347] Connected to a master server at 127.30.76.126:43607
I20260808 15:54:25.815341 18583 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:25.815951 18583 heartbeater.cc:511] Master 127.30.76.126:43607 requested a full tablet report, sending...
I20260808 15:54:25.817030 18108 ts_manager.cc:194] Registered new tserver with Master: 8d30331485054a299246450a432e7c1e (127.30.76.66:38843)
I20260808 15:54:25.817607 18108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:40095
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:54:25.838390 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:43607
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:25.948776 18588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:25.949038 18588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:25.949112 18588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:25.952975 18588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:25.953059 18588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:25.953150 18588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:54:25.958287 18588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:25.959242 18588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:25.960215 18588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:25.969462 18598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.969467 18597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:25.969477 18601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:25.969820 18588 server_base.cc:1034] running on GCE node
I20260808 15:54:25.970223 18588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:25.970836 18588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:25.972018 18588 hybrid_clock.cc:648] HybridClock initialized: now 1786204465972006 us; error 27 us; skew 500 ppm
Aug 08 15:54:25 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:25.974642 18588 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:54:25.975431 18588 webserver.cc:460] Webserver started at http://127.30.76.67:44587/ using document root <none> and password file <none>
I20260808 15:54:25.976020 18588 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:25.976109 18588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:25.976387 18588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:25.978407 18588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "91710133d37fa1909da74057cb0d8b72"
server_key_iv: "5c1c123f299b1c2265bdb0342661dbbd"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.978955 18588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "91710133d37fa1909da74057cb0d8b72"
server_key_iv: "5c1c123f299b1c2265bdb0342661dbbd"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.982429 18588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:54:25.984666 18613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:25.985774 18588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:54:25.985903 18588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "91710133d37fa1909da74057cb0d8b72"
server_key_iv: "5c1c123f299b1c2265bdb0342661dbbd"
server_key_version: "encryptionkey@0"
I20260808 15:54:25.986007 18588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:25.999568 18588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:26.002328 18588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:26.002516 18588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:26.003181 18588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:26.004210 18588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:26.004281 18588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.004351 18588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:26.004383 18588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.013080 18588 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:38447
I20260808 15:54:26.013185 18764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:38447 every 8 connection(s)
I20260808 15:54:26.014206 18588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204465, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:26.024535 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 18588
I20260808 15:54:26.024698 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:54:26.025090 31025 external_mini_cluster.cc:1428] Setting key bb5b2b19f9558bbab78d6a7de127a158
I20260808 15:54:26.028021 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.015971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37977 (local address 127.30.76.126:43607)
0808 15:54:26.016108 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.016111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.016899 (+ 788us) server_negotiation.cc:407] Connection header received
0808 15:54:26.018202 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.018205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.018266 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.018372 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:26.019496 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.020066 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.021189 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.021417 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.021529 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.023931 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:26.023934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:26.023937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:26.023960 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:26.025743 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.026375 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.026379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.026381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.026493 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.026938 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.026941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.026942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.027130 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:26.027216 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.027804 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.027907 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:54:26.028816 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.016319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:37977)
0808 15:54:26.016743 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.016783 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.017989 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.018535 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.018542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.018886 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:26.019351 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.019364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.020199 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.020209 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.021058 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.021068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.021577 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.021581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.021743 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.022254 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:26.022269 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:26.023763 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:26.025890 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.025896 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.025909 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.026266 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.026618 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.026621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.026623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.026825 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.027247 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:26.027253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:26.027557 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.027744 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.027975 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":114,"threads_started":1}
I20260808 15:54:26.029838 18765 heartbeater.cc:347] Connected to a master server at 127.30.76.126:43607
I20260808 15:54:26.030084 18765 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:26.030545 18765 heartbeater.cc:511] Master 127.30.76.126:43607 requested a full tablet report, sending...
I20260808 15:54:26.031517 18108 ts_manager.cc:194] Registered new tserver with Master: 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
I20260808 15:54:26.032090 18108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:37977
I20260808 15:54:26.039278 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:54:26.048978 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.041404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59368 (local address 127.30.76.126:43607)
0808 15:54:26.041541 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.041547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.041736 (+ 189us) server_negotiation.cc:407] Connection header received
0808 15:54:26.041891 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.041894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.041953 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.042038 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:26.042773 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.043304 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.044236 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.044471 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.044579 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.045896 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:26.045900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:26.045902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:26.045929 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:26.047156 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.047670 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.047673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.047675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.047767 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.048061 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.048065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.048066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.048286 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:26.048440 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.048680 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.048799 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:54:26.051796 18108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:54:26.054870 18108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:54:26.069489 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.064109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.30.76.67:38447)
0808 15:54:26.064805 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.064817 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.064837 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:54:26.064901 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.064905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.065061 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.065199 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:26.065557 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.066759 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.068261 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.068801 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.068925 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.069074 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.069234 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.069337 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":89,"threads_started":1}
I20260808 15:54:26.069489 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.063482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38447 (local address 127.0.0.1:57076)
0808 15:54:26.064050 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.064063 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.064573 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.065202 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.065208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.065231 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:26.065422 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.065433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.066901 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.066905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.068139 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.068148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.068926 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.068929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.069032 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.069067 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.069202 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.069346 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":97,"spinlock_wait_cycles":924928,"thread_start_us":115,"threads_started":1}
I20260808 15:54:26.070185 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.064053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38843 (local address 127.0.0.1:47440)
0808 15:54:26.064304 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.064318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.064424 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.065561 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.065564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.065580 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:26.065664 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.065668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.066633 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.066645 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.068174 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.068186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.069468 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.069472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.069839 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.069868 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.069953 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.070025 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":89,"thread_start_us":105,"threads_started":1}
I20260808 15:54:26.070228 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.064492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.30.76.66:38843)
0808 15:54:26.064929 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.064941 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.064964 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:54:26.065042 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.065049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.065244 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.065396 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:26.065780 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.066506 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.068332 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.069152 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.069264 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.069400 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.070006 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.070103 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":109,"threads_started":1}
I20260808 15:54:26.070217 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.063285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42031 (local address 127.0.0.1:58974)
0808 15:54:26.063832 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.063878 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.064426 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.065292 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.065296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.065313 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:26.065448 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.065453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.067070 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.067073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.068208 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.068219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.069468 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.069472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.069878 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.069897 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.069990 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.070058 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":300,"thread_start_us":89,"threads_started":1}
I20260808 15:54:26.070724 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.063451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58974 (local address 127.30.76.65:42031)
0808 15:54:26.063889 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.063907 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.064297 (+ 390us) server_negotiation.cc:407] Connection header received
0808 15:54:26.064732 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.064740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.064945 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.065094 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:26.065601 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.066761 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.068340 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.069160 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.069264 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.069400 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.070319 (+ 919us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.070586 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":140,"threads_started":1}
I20260808 15:54:26.072777 18677 tablet_service.cc:1467] Processing CreateTablet for tablet ff944ceec15f4c39ab0407580c983e14 (DEFAULT_TABLE table=test-table [id=8292db7ed98e432a9bc39aaeb023d66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:26.073180 18296 tablet_service.cc:1467] Processing CreateTablet for tablet ff944ceec15f4c39ab0407580c983e14 (DEFAULT_TABLE table=test-table [id=8292db7ed98e432a9bc39aaeb023d66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:26.073248 18489 tablet_service.cc:1467] Processing CreateTablet for tablet ff944ceec15f4c39ab0407580c983e14 (DEFAULT_TABLE table=test-table [id=8292db7ed98e432a9bc39aaeb023d66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:26.073966 18677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff944ceec15f4c39ab0407580c983e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:26.074111 18296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff944ceec15f4c39ab0407580c983e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:26.074184 18489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff944ceec15f4c39ab0407580c983e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:26.080603 18787 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Bootstrap starting.
I20260808 15:54:26.081571 18791 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Bootstrap starting.
I20260808 15:54:26.083361 18787 tablet_bootstrap.cc:654] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:26.084292 18791 tablet_bootstrap.cc:654] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:26.084904 18787 log.cc:824] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:26.085035 18791 log.cc:824] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:26.085559 18790 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Bootstrap starting.
I20260808 15:54:26.087072 18791 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: No bootstrap required, opened a new log
I20260808 15:54:26.087294 18791 ts_tablet_manager.cc:1397] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:54:26.088137 18790 tablet_bootstrap.cc:654] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:26.088801 18787 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: No bootstrap required, opened a new log
I20260808 15:54:26.089041 18787 ts_tablet_manager.cc:1397] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 15:54:26.089042 18790 log.cc:824] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:26.091003 18790 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: No bootstrap required, opened a new log
I20260808 15:54:26.091183 18790 ts_tablet_manager.cc:1397] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 15:54:26.091480 18791 raft_consensus.cc:348] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.091758 18791 raft_consensus.cc:374] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:26.091846 18791 raft_consensus.cc:729] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, State: Initialized, Role: FOLLOWER
I20260808 15:54:26.092312 18791 consensus_queue.cc:260] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.092586 18787 raft_consensus.cc:348] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.092885 18787 raft_consensus.cc:374] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:26.092980 18765 heartbeater.cc:503] Master 127.30.76.126:43607 was elected leader, sending a full tablet report...
I20260808 15:54:26.092994 18787 raft_consensus.cc:729] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629015e49462454988aa207e71601766, State: Initialized, Role: FOLLOWER
I20260808 15:54:26.093303 18791 ts_tablet_manager.cc:1428] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:54:26.093551 18787 consensus_queue.cc:260] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.094543 18787 ts_tablet_manager.cc:1428] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 15:54:26.094916 18385 heartbeater.cc:503] Master 127.30.76.126:43607 was elected leader, sending a full tablet report...
I20260808 15:54:26.095021 18790 raft_consensus.cc:348] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.095252 18790 raft_consensus.cc:374] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:26.095352 18790 raft_consensus.cc:729] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d30331485054a299246450a432e7c1e, State: Initialized, Role: FOLLOWER
I20260808 15:54:26.095810 18790 consensus_queue.cc:260] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.096455 18583 heartbeater.cc:503] Master 127.30.76.126:43607 was elected leader, sending a full tablet report...
I20260808 15:54:26.096776 18790 ts_tablet_manager.cc:1428] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:54:26.097115 18801 raft_consensus.cc:482] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:26.097594 18801 raft_consensus.cc:504] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.098852 18801 leader_election.cc:288] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447), 8d30331485054a299246450a432e7c1e (127.30.76.66:38843)
I20260808 15:54:26.102770 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.099137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:50829 (local address 127.30.76.66:38843)
0808 15:54:26.099273 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.099278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.099573 (+ 295us) server_negotiation.cc:407] Connection header received
0808 15:54:26.099690 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.099693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.099744 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.099798 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:26.100115 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.100593 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.101799 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.102236 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.102367 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.102417 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.102588 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.102660 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:54:26.102787 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.099074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38843 (local address 127.30.76.65:50829)
0808 15:54:26.099443 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.099456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.099565 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.099908 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.099910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.099923 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:26.100009 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.100013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.100718 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.100721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.101678 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.101688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.102361 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.102365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.102467 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.102486 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.102573 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.102643 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":43,"thread_start_us":104,"threads_started":1}
I20260808 15:54:26.102905 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.099295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39305 (local address 127.30.76.67:38447)
0808 15:54:26.099385 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.099389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.099409 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:54:26.099470 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.099471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.099531 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.099637 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:26.099925 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.100663 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.101928 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.102318 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.102453 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.102504 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.102702 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.102771 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 15:54:26.103231 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.099047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38447 (local address 127.30.76.65:39305)
0808 15:54:26.099283 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.099299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.099385 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.099648 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.099651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.099669 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:26.099790 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.099797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.100777 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.100780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.101793 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.101803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.102446 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.102450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.102569 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.102596 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.102686 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.102758 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":19}
I20260808 15:54:26.103255 18516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff944ceec15f4c39ab0407580c983e14" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d30331485054a299246450a432e7c1e" is_pre_election: true
I20260808 15:54:26.103493 18516 raft_consensus.cc:2454] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 0.
I20260808 15:54:26.103847 18706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff944ceec15f4c39ab0407580c983e14" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" is_pre_election: true
I20260808 15:54:26.103983 18236 leader_election.cc:302] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629015e49462454988aa207e71601766, 8d30331485054a299246450a432e7c1e; no voters:
I20260808 15:54:26.104058 18706 raft_consensus.cc:2454] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 0.
I20260808 15:54:26.104257 18801 raft_consensus.cc:2781] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:54:26.104346 18801 raft_consensus.cc:482] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:54:26.104406 18801 raft_consensus.cc:3037] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:26.105334 18801 raft_consensus.cc:504] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.105659 18801 leader_election.cc:288] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 election: Requested vote from peers 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447), 8d30331485054a299246450a432e7c1e (127.30.76.66:38843)
I20260808 15:54:26.106005 18516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff944ceec15f4c39ab0407580c983e14" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d30331485054a299246450a432e7c1e"
I20260808 15:54:26.106006 18706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff944ceec15f4c39ab0407580c983e14" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
I20260808 15:54:26.106114 18706 raft_consensus.cc:3037] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:26.106123 18516 raft_consensus.cc:3037] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:26.107043 18516 raft_consensus.cc:2454] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 1.
I20260808 15:54:26.107052 18706 raft_consensus.cc:2454] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 1.
I20260808 15:54:26.107339 18236 leader_election.cc:302] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629015e49462454988aa207e71601766, 8d30331485054a299246450a432e7c1e; no voters:
I20260808 15:54:26.107589 18801 raft_consensus.cc:2781] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:26.107813 18801 raft_consensus.cc:686] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 LEADER]: Becoming Leader. State: Replica: 629015e49462454988aa207e71601766, State: Running, Role: LEADER
I20260808 15:54:26.108115 18801 consensus_queue.cc:237] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.111027 18107 catalog_manager.cc:5495] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 reported cstate change: term changed from 0 to 1, leader changed from <none> to 629015e49462454988aa207e71601766 (127.30.76.65). New cstate: current_term: 1 leader_uuid: "629015e49462454988aa207e71601766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } health_report { overall_health: UNKNOWN } } }
I20260808 15:54:26.127991 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.124683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.30.76.65:42031)
0808 15:54:26.124826 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.124830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.124842 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:54:26.124963 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.124965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.125015 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.125099 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:54:26.125290 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.125756 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.126579 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.126883 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.126962 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.127214 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:54:26.127655 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:54:26.127746 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.127835 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.127881 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:54:26.133451 18706 raft_consensus.cc:1261] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Refusing update from remote peer 629015e49462454988aa207e71601766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:26.133452 18516 raft_consensus.cc:1261] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Refusing update from remote peer 629015e49462454988aa207e71601766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:26.134168 18801 consensus_queue.cc:1035] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:26.134387 18809 consensus_queue.cc:1035] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:26.142603 18818 mvcc.cc:205] Tried to move back new op lower bound from 7316293493275574272 to 7316293493181566976. Current Snapshot: MvccSnapshot[applied={T|T < 7316293493275574272}]
I20260808 15:54:26.143766 18819 mvcc.cc:205] Tried to move back new op lower bound from 7316293493275574272 to 7316293493181566976. Current Snapshot: MvccSnapshot[applied={T|T < 7316293493275574272}]
I20260808 15:54:26.152091 18107 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:26.152236 18107 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:26.154646 18107 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Sending DeleteTablet for 3 replicas of tablet ff944ceec15f4c39ab0407580c983e14
I20260808 15:54:26.155419 18296 tablet_service.cc:1514] Processing DeleteTablet for tablet ff944ceec15f4c39ab0407580c983e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:26 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:58974
I20260808 15:54:26.155457 18677 tablet_service.cc:1514] Processing DeleteTablet for tablet ff944ceec15f4c39ab0407580c983e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:26 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:57076
I20260808 15:54:26.155633 18489 tablet_service.cc:1514] Processing DeleteTablet for tablet ff944ceec15f4c39ab0407580c983e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:26 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:47440
I20260808 15:54:26.155999 18831 tablet_replica.cc:330] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: stopping tablet replica
I20260808 15:54:26.156272 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 18203
I20260808 15:54:26.156281 18831 raft_consensus.cc:2229] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:54:26.160005 18830 tablet_replica.cc:330] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: stopping tablet replica
I20260808 15:54:26.160317 18830 raft_consensus.cc:2229] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:26.160629 18830 raft_consensus.cc:2258] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:26.161263 18834 tablet_replica.cc:330] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: stopping tablet replica
I20260808 15:54:26.161538 18834 raft_consensus.cc:2229] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:26.161800 18834 raft_consensus.cc:2258] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:26.163391 18830 ts_tablet_manager.cc:1905] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:26.164022 18834 ts_tablet_manager.cc:1905] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:26.166327 18830 ts_tablet_manager.cc:1918] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:26.166441 18830 log.cc:1196] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/ff944ceec15f4c39ab0407580c983e14
I20260808 15:54:26.166736 18834 ts_tablet_manager.cc:1918] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:26.166791 18830 ts_tablet_manager.cc:1939] T ff944ceec15f4c39ab0407580c983e14 P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Deleting consensus metadata
I20260808 15:54:26.166838 18834 log.cc:1196] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/ff944ceec15f4c39ab0407580c983e14
I20260808 15:54:26.167164 18834 ts_tablet_manager.cc:1939] T ff944ceec15f4c39ab0407580c983e14 P 8d30331485054a299246450a432e7c1e: Deleting consensus metadata
I20260808 15:54:26.168119 18093 catalog_manager.cc:4841] TS 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): tablet ff944ceec15f4c39ab0407580c983e14 (table test-table [id=8292db7ed98e432a9bc39aaeb023d66e]) successfully deleted
I20260808 15:54:26.168211 18092 catalog_manager.cc:4841] TS 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): tablet ff944ceec15f4c39ab0407580c983e14 (table test-table [id=8292db7ed98e432a9bc39aaeb023d66e]) successfully deleted
W20260808 15:54:26.170881 18091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:42031 (error 108)
I20260808 15:54:26.171012 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 18392
I20260808 15:54:26.178488 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.178182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42031 (local address 127.0.0.1:58994)
0808 15:54:26.178320 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.178401 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 15:54:26.178761 18091 catalog_manager.cc:4584] TS 629015e49462454988aa207e71601766 (127.30.76.65:42031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff944ceec15f4c39ab0407580c983e14: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42031: connect: Connection refused (error 111)
I20260808 15:54:26.183470 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 18588
I20260808 15:54:26.198264 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 18062
I20260808 15:54:26.211329 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:43607
--webserver_interface=127.30.76.126
--webserver_port=34363
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:26.321410 18836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:26.321687 18836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:26.321763 18836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:26.325812 18836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:26.325883 18836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:26.325940 18836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:26.325975 18836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:26.326011 18836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:26.331072 18836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=34363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:26.332062 18836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:26.333038 18836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:26.347100 18842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.347111 18847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.347113 18844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:26.347388 18836 server_base.cc:1034] running on GCE node
I20260808 15:54:26.348116 18836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:26.349231 18836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:26.350420 18836 hybrid_clock.cc:648] HybridClock initialized: now 1786204466350404 us; error 35 us; skew 500 ppm
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:26.353200 18836 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:26.354012 18836 webserver.cc:460] Webserver started at http://127.30.76.126:34363/ using document root <none> and password file <none>
I20260808 15:54:26.354538 18836 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:26.354609 18836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:26.358371 18836 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:26.360446 18861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.361634 18836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:26.361747 18836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "81cdafedf1024c639557bd6e95fe074b"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "05671ca2f7fd17b5a610e76eabc81409"
server_key_iv: "18a4661b34dc0f647c1e39ae6671ad5e"
server_key_version: "encryptionkey@0"
I20260808 15:54:26.362303 18836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:26.382231 18836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:26.385028 18836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:26.385187 18836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:26.391945 18836 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:43607
I20260808 15:54:26.392217 18936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:43607 every 8 connection(s)
I20260808 15:54:26.393038 18836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:26.397358 18937 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:54:26.397465 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 18836
I20260808 15:54:26.397912 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:42031
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:43607
--webserver_port=45099
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:26.399087 18937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap starting.
I20260808 15:54:26.416514 18937 log.cc:824] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:26.425107 18937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:54:26.425484 18937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap complete.
I20260808 15:54:26.429695 18937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:26.430071 18937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cdafedf1024c639557bd6e95fe074b, State: Initialized, Role: FOLLOWER
I20260808 15:54:26.430594 18937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:26.430714 18937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:26.430804 18937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:26.430920 18937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:54:26.432874 18937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:26.433212 18937 leader_election.cc:302] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cdafedf1024c639557bd6e95fe074b; no voters:
I20260808 15:54:26.433543 18937 leader_election.cc:288] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:54:26.433630 18942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:54:26.433820 18942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 LEADER]: Becoming Leader. State: Replica: 81cdafedf1024c639557bd6e95fe074b, State: Running, Role: LEADER
I20260808 15:54:26.434108 18942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:26.434787 18937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:26.435099 18943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "81cdafedf1024c639557bd6e95fe074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } } }
I20260808 15:54:26.435089 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cdafedf1024c639557bd6e95fe074b. Latest consensus state: current_term: 2 leader_uuid: "81cdafedf1024c639557bd6e95fe074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } } }
I20260808 15:54:26.435248 18943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:26.435267 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:26.435652 18947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:26.440482 18947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:26.441115 18947 catalog_manager.cc:1259] Loaded cluster ID: c89494ae0f564f01951ea0d6f14c6136
I20260808 15:54:26.441277 18947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:26.442716 18947 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 15:54:26.447378 18947 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 15:54:26.452512 18947 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:26.453217 18947 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Loaded TSK: 0
I20260808 15:54:26.454438 18947 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:54:26.516813 18939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:26.517066 18939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:26.517107 18939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:26.521212 18939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:26.521272 18939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:26.521328 18939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:54:26.526551 18939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:42031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=45099
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:26.527681 18939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:26.528677 18939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:26.539547 18978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:26.539748 18939 server_base.cc:1034] running on GCE node
W20260808 15:54:26.539554 18976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.540004 18980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:26.540508 18939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:26.541194 18939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:26.542414 18939 hybrid_clock.cc:648] HybridClock initialized: now 1786204466542387 us; error 44 us; skew 500 ppm
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:26.545284 18939 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:54:26.546185 18939 webserver.cc:460] Webserver started at http://127.30.76.65:45099/ using document root <none> and password file <none>
I20260808 15:54:26.546741 18939 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:26.546846 18939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:26.550724 18939 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:26.552981 18990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.554253 18939 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:54:26.554386 18939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "629015e49462454988aa207e71601766"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "14e8e0ac1575f9dc58e4e630143f97fc"
server_key_iv: "bd8c5407a2387751f6cafde06eb343da"
server_key_version: "encryptionkey@0"
I20260808 15:54:26.554908 18939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:26.579734 18939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:26.582592 18939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:26.582803 18939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:26.583406 18939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:26.584498 18998 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 15:54:26.588581 18939 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 15:54:26.588641 18939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 15:54:26.588717 18939 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 15:54:26.590288 18939 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 15:54:26.590330 18939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 15:54:26.590600 18998 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Bootstrap starting.
I20260808 15:54:26.600912 18939 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:42031
I20260808 15:54:26.601001 19159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:42031 every 8 connection(s)
I20260808 15:54:26.601826 18939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:54:26.605923 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 18939
I20260808 15:54:26.606472 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:38843
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:43607
--webserver_port=36407
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:26.610078 18998 log.cc:824] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:26.617920 18998 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:54:26.618336 18998 tablet_bootstrap.cc:492] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Bootstrap complete.
I20260808 15:54:26.619728 18998 ts_tablet_manager.cc:1397] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.015s
I20260808 15:54:26.621966 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.603886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:36555 (local address 127.30.76.126:43607)
0808 15:54:26.604233 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.604254 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.604598 (+ 344us) server_negotiation.cc:407] Connection header received
0808 15:54:26.605714 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.605732 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.606134 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.606495 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:26.608285 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.609740 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.611601 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.612272 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.612457 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.615780 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:26.615790 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:26.615811 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:26.615854 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:26.618249 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.618962 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.618971 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.618980 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.619149 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.619591 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.619594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.619596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.620089 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:26.620226 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.620836 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.621141 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":120,"threads_started":1}
I20260808 15:54:26.621994 19164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.603940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:36555)
0808 15:54:26.604439 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.604479 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.605274 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.606809 (+ 1535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.606819 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.607295 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:26.608061 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.608083 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.609929 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.609936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.611428 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.611445 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.612437 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.612442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.612739 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.613504 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:26.613532 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:26.615518 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:26.618427 (+ 2909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.618433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.618446 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.618810 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.619260 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.619266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.619271 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.619449 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.620217 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:26.620223 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:26.620492 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.620683 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.621020 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":3,"thread_start_us":166,"threads_started":1}
I20260808 15:54:26.622598 18998 raft_consensus.cc:348] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.623114 18998 raft_consensus.cc:729] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629015e49462454988aa207e71601766, State: Initialized, Role: FOLLOWER
I20260808 15:54:26.623647 18998 consensus_queue.cc:260] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } }
I20260808 15:54:26.624693 18998 ts_tablet_manager.cc:1428] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:54:26.624964 19160 heartbeater.cc:347] Connected to a master server at 127.30.76.126:43607
I20260808 15:54:26.625177 19160 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:26.625651 19160 heartbeater.cc:511] Master 127.30.76.126:43607 requested a full tablet report, sending...
I20260808 15:54:26.628170 18883 ts_manager.cc:194] Registered new tserver with Master: 629015e49462454988aa207e71601766 (127.30.76.65:42031)
I20260808 15:54:26.632046 18883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:36555
I20260808 15:54:26.634588 19160 heartbeater.cc:503] Master 127.30.76.126:43607 was elected leader, sending a full tablet report...
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:54:26.641860 19177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.631349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42031 (local address 127.0.0.1:59006)
0808 15:54:26.632037 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.632070 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.632225 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.632690 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.632695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.633135 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:26.633235 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.633242 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.634210 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.634215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.635120 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.635130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.635971 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.635975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.636077 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.636676 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:26.636691 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:26.638114 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:26.640165 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.640173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.640176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.640387 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.640712 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.640714 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.640715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.640789 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.641315 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:26.641319 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:26.641415 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.641522 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.641662 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":107,"threads_started":1}
I20260808 15:54:26.641935 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.631391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.30.76.65:42031)
0808 15:54:26.631799 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.631810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.632232 (+ 422us) server_negotiation.cc:407] Connection header received
0808 15:54:26.632392 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.632397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.632540 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.632677 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:26.633357 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.634084 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.635294 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.635847 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.635955 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.638250 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:26.638259 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:26.638271 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:26.638297 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:26.640060 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.640498 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.640506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.640510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.640612 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.640904 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.640907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.640909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.641196 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:26.641319 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.641623 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.641785 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":320,"thread_start_us":84,"threads_started":1}
I20260808 15:54:26.642983 19058 tablet_service.cc:1514] Processing DeleteTablet for tablet ff944ceec15f4c39ab0407580c983e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:26 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:59006
I20260808 15:54:26.643929 19178 tablet_replica.cc:330] stopping tablet replica
I20260808 15:54:26.644209 19178 raft_consensus.cc:2229] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:26.644435 19178 raft_consensus.cc:2258] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:26.645323 19178 ts_tablet_manager.cc:1905] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:26.648686 19178 ts_tablet_manager.cc:1918] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:26.648802 19178 log.cc:1196] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/ff944ceec15f4c39ab0407580c983e14
I20260808 15:54:26.649076 19178 ts_tablet_manager.cc:1939] T ff944ceec15f4c39ab0407580c983e14 P 629015e49462454988aa207e71601766: Deleting consensus metadata
I20260808 15:54:26.650604 18866 catalog_manager.cc:4841] TS 629015e49462454988aa207e71601766 (127.30.76.65:42031): tablet ff944ceec15f4c39ab0407580c983e14 (table test-table [id=8292db7ed98e432a9bc39aaeb023d66e]) successfully deleted
W20260808 15:54:26.726208 19165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:26.726442 19165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:26.726483 19165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:26.730482 19165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:26.730542 19165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:26.730597 19165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:54:26.735956 19165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:38843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=36407
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:26.736956 19165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:26.737884 19165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:26.747380 19194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.747423 19189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.747431 19187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:26.747812 19165 server_base.cc:1034] running on GCE node
I20260808 15:54:26.748234 19165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:26.748874 19165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:26.750090 19165 hybrid_clock.cc:648] HybridClock initialized: now 1786204466750071 us; error 35 us; skew 500 ppm
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:26.753095 19165 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:54:26.754069 19165 webserver.cc:460] Webserver started at http://127.30.76.66:36407/ using document root <none> and password file <none>
I20260808 15:54:26.754679 19165 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:26.754798 19165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:26.758651 19165 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:26.760972 19203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.762214 19165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:54:26.762319 19165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "8d30331485054a299246450a432e7c1e"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "7d1ea9430f53507851f8cd987b164595"
server_key_iv: "ea35b71afca932c90df309f810d8a86b"
server_key_version: "encryptionkey@0"
I20260808 15:54:26.762804 19165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:26.791047 19165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:26.794687 19165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:26.794901 19165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:26.795562 19165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:26.796437 19165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:26.796487 19165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.796557 19165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:26.796591 19165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.805338 19165 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:38843
I20260808 15:54:26.805369 19382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:38843 every 8 connection(s)
I20260808 15:54:26.806336 19165 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:54:26.814450 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 19165
I20260808 15:54:26.814899 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:38447
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:43607
--webserver_port=44587
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:26.819988 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.808142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:50895 (local address 127.30.76.126:43607)
0808 15:54:26.808283 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:26.808286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:26.809133 (+ 847us) server_negotiation.cc:407] Connection header received
0808 15:54:26.810493 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:26.810497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:26.810558 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:26.810660 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:26.811783 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.812316 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.813424 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.813665 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.813772 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.816484 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:26.816487 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:26.816489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:26.816512 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:26.817978 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.818490 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.818493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.818495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.818575 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:26.818909 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:26.818914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:26.818916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:26.819088 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:26.819210 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:26.819687 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 15:54:26.819815 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:54:26.820724 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:26.808557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:50895)
0808 15:54:26.808982 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:26.809021 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:26.810280 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:26.810833 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:26.810840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:26.811168 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:26.811635 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.811649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.812442 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.812446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.813290 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:26.813299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:26.813819 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:26.813824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:26.813978 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:26.814563 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:26.814607 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:26.816311 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:26.818110 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.818116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.818128 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.818375 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.818672 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:26.818676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:26.818677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:26.818812 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:26.819207 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:26.819213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:26.819439 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 15:54:26.819652 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 15:54:26.819942 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":121,"threads_started":1}
I20260808 15:54:26.821892 19384 heartbeater.cc:347] Connected to a master server at 127.30.76.126:43607
I20260808 15:54:26.822160 19384 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:26.822762 19384 heartbeater.cc:511] Master 127.30.76.126:43607 requested a full tablet report, sending...
I20260808 15:54:26.823976 18883 ts_manager.cc:194] Registered new tserver with Master: 8d30331485054a299246450a432e7c1e (127.30.76.66:38843)
I20260808 15:54:26.824874 18883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:50895
W20260808 15:54:26.929913 19388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:26.930181 19388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:26.930224 19388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:26.934196 19388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:26.934250 19388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:26.934306 19388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:54:26.939450 19388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:38447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=44587
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:26.940466 19388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:26.941555 19388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:26.951083 19395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.951056 19399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:26.951074 19396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:26.951416 19388 server_base.cc:1034] running on GCE node
I20260808 15:54:26.951937 19388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:26.952545 19388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:26.953744 19388 hybrid_clock.cc:648] HybridClock initialized: now 1786204466953729 us; error 26 us; skew 500 ppm
Aug 08 15:54:26 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:26.956700 19388 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:54:26.957487 19388 webserver.cc:460] Webserver started at http://127.30.76.67:44587/ using document root <none> and password file <none>
I20260808 15:54:26.958017 19388 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:26.958168 19388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:26.961935 19388 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:54:26.963871 19409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:26.965144 19388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:54:26.965258 19388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "91710133d37fa1909da74057cb0d8b72"
server_key_iv: "5c1c123f299b1c2265bdb0342661dbbd"
server_key_version: "encryptionkey@0"
I20260808 15:54:26.965787 19388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:27.002756 19388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:27.006201 19388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:27.006378 19388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:27.006934 19388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:27.007755 19388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:27.007805 19388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.007845 19388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:27.007866 19388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.016754 19388 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:38447
I20260808 15:54:27.016794 19590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:38447 every 8 connection(s)
I20260808 15:54:27.017743 19388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:54:27.022693 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 19388
Aug 08 15:54:27 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204466, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:27.032546 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.023430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.30.76.126:43607)
0808 15:54:27.023737 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.023742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.023940 (+ 198us) server_negotiation.cc:407] Connection header received
0808 15:54:27.024102 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.024110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.024151 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.024267 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:27.025034 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.025744 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.026622 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.026855 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.026965 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.028400 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:27.028406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:27.028412 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:27.028444 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:27.030605 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:27.031090 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:27.031093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:27.031094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:27.031221 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:27.031586 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:27.031589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:27.031590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:27.031832 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:27.032060 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.032240 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.032341 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
I20260808 15:54:27.033674 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.020087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:44877)
0808 15:54:27.020566 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.020609 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:27.021831 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:27.022341 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:27.022347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:27.022719 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:27.023356 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.023379 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.024244 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.024251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.025504 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.025515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.025992 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.025997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.026189 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.026763 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:27.026779 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:27.028348 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:27.030739 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:27.030744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:27.030758 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:27.031082 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:27.031390 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:27.031394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:27.031396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:27.031600 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:27.031957 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:27.031963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:27.032261 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 15:54:27.032457 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 15:54:27.032810 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":127,"threads_started":1}
I20260808 15:54:27.034338 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.019612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:44877 (local address 127.30.76.126:43607)
0808 15:54:27.019761 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.019767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.020734 (+ 967us) server_negotiation.cc:407] Connection header received
0808 15:54:27.022044 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.022046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.022095 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.022183 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:27.023544 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.024106 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.025629 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.025838 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.025954 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.028530 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:27.028536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:27.028539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:27.028578 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:27.030581 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:27.031215 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:27.031218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:27.031219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:27.031294 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:27.031698 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:27.031701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:27.031703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:27.031854 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:27.031943 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.034146 (+ 2203us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.034238 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:54:27.035175 19591 heartbeater.cc:347] Connected to a master server at 127.30.76.126:43607
I20260808 15:54:27.035427 19591 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:27.035945 19591 heartbeater.cc:511] Master 127.30.76.126:43607 requested a full tablet report, sending...
I20260808 15:54:27.036911 18883 ts_manager.cc:194] Registered new tserver with Master: 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
I20260808 15:54:27.037469 18883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:44877
I20260808 15:54:27.046043 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:54:27.050058 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.046647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59378 (local address 127.30.76.126:43607)
0808 15:54:27.046760 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.046763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.046914 (+ 151us) server_negotiation.cc:407] Connection header received
0808 15:54:27.047025 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.047027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.047113 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.047205 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:54:27.047406 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.047933 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.048928 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.049133 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.049206 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.049463 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:54:27.049777 (+ 314us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:54:27.049859 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.049914 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.049952 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:54:27.050704 18883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:54:27.053915 18883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:54:27.061988 19058 tablet_service.cc:1467] Processing CreateTablet for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b (DEFAULT_TABLE table=test-table [id=3f7f014da330487481a9c051bb607488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:27.062840 19058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:27.065956 18998 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Bootstrap starting.
I20260808 15:54:27.066715 19177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.060874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38843 (local address 127.0.0.1:47450)
0808 15:54:27.061009 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.061023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:27.061109 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:27.062120 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:27.062122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:27.062140 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:27.062246 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.062252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.063880 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.063883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.065051 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.065063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.066188 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.066191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.066288 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.066408 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 15:54:27.066480 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 15:54:27.066559 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":47}
I20260808 15:54:27.066861 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.061106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.30.76.66:38843)
0808 15:54:27.061620 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.061634 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.061659 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:54:27.061736 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.061741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.061947 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.062282 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:27.062458 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.063714 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.065218 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.066054 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.066352 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.066569 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.066628 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.066713 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":126,"threads_started":1}
I20260808 15:54:27.066890 18998 tablet_bootstrap.cc:654] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:27.068728 18998 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: No bootstrap required, opened a new log
I20260808 15:54:27.068820 18998 ts_tablet_manager.cc:1397] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:54:27.069314 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.061220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38447 (local address 127.0.0.1:57092)
0808 15:54:27.063848 (+ 2628us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.063863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:27.063961 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:27.064370 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:27.064373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:27.064391 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:27.064478 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.064484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.066200 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.066204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.067467 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.067484 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.068869 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.068872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.069007 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.069038 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:54:27.069111 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 15:54:27.069182 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2473,"mutex_wait_us":21,"thread_start_us":113,"threads_started":1}
I20260808 15:54:27.069267 18998 raft_consensus.cc:348] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.069407 18998 raft_consensus.cc:374] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:27.069361 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.061632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.30.76.67:38447)
0808 15:54:27.061921 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.061932 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.063942 (+ 2010us) server_negotiation.cc:407] Connection header received
0808 15:54:27.064066 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.064071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.064239 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.064390 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:27.064573 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.065996 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.067683 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.068765 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.068873 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.069012 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.069145 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.069227 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":122,"threads_started":1}
I20260808 15:54:27.069461 18998 raft_consensus.cc:729] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629015e49462454988aa207e71601766, State: Initialized, Role: FOLLOWER
I20260808 15:54:27.069614 18998 consensus_queue.cc:260] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.070399 18998 ts_tablet_manager.cc:1428] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 15:54:27.070027 19279 tablet_service.cc:1467] Processing CreateTablet for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b (DEFAULT_TABLE table=test-table [id=3f7f014da330487481a9c051bb607488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:27.071096 19279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:27.071921 19493 tablet_service.cc:1467] Processing CreateTablet for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b (DEFAULT_TABLE table=test-table [id=3f7f014da330487481a9c051bb607488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:27.072921 19493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:27.077823 19610 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Bootstrap starting.
I20260808 15:54:27.079183 19611 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Bootstrap starting.
I20260808 15:54:27.080291 19610 tablet_bootstrap.cc:654] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:27.080973 19611 tablet_bootstrap.cc:654] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:27.081065 19610 log.cc:824] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:27.082024 19611 log.cc:824] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:27.083068 19610 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: No bootstrap required, opened a new log
I20260808 15:54:27.083263 19610 ts_tablet_manager.cc:1397] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:54:27.084017 19611 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: No bootstrap required, opened a new log
I20260808 15:54:27.084204 19611 ts_tablet_manager.cc:1397] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:54:27.087132 19610 raft_consensus.cc:348] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.087407 19610 raft_consensus.cc:374] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:27.087491 19610 raft_consensus.cc:729] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d30331485054a299246450a432e7c1e, State: Initialized, Role: FOLLOWER
I20260808 15:54:27.087956 19610 consensus_queue.cc:260] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.087939 19611 raft_consensus.cc:348] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.088188 19611 raft_consensus.cc:374] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:27.088260 19611 raft_consensus.cc:729] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, State: Initialized, Role: FOLLOWER
I20260808 15:54:27.088649 19384 heartbeater.cc:503] Master 127.30.76.126:43607 was elected leader, sending a full tablet report...
I20260808 15:54:27.088673 19611 consensus_queue.cc:260] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.089011 19610 ts_tablet_manager.cc:1428] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:54:27.089332 19591 heartbeater.cc:503] Master 127.30.76.126:43607 was elected leader, sending a full tablet report...
I20260808 15:54:27.089661 19611 ts_tablet_manager.cc:1428] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 15:54:27.103060 19161 tablet.cc:2369] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:54:27.235458 19174 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:27.235788 19174 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.237056 19174 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
I20260808 15:54:27.241315 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.237474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:35205 (local address 127.30.76.67:38447)
0808 15:54:27.237611 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.237617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.237876 (+ 259us) server_negotiation.cc:407] Connection header received
0808 15:54:27.238030 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.238033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.238083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.238167 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:27.238417 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.238910 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.240147 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.240642 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.240775 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.240827 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.241079 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.241157 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:54:27.241469 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.237279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38843 (local address 127.30.76.65:47369)
0808 15:54:27.237667 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.237680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:27.237832 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:27.238127 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:27.238130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:27.238147 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:27.238272 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.238277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.239001 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.239005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.240086 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.240101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.241068 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.241071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.241158 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.241180 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:54:27.241254 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 15:54:27.241316 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":65,"thread_start_us":88,"threads_started":1}
I20260808 15:54:27.241544 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.237387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:47369 (local address 127.30.76.66:38843)
0808 15:54:27.237522 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.237527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.237771 (+ 244us) server_negotiation.cc:407] Connection header received
0808 15:54:27.237968 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.237971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.238027 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.238116 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:27.238387 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.238888 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.240225 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.240894 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.241045 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.241117 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.241288 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.241377 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:54:27.242017 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.237378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38447 (local address 127.30.76.65:35205)
0808 15:54:27.237788 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.237800 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:27.237918 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:27.238189 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:27.238192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:27.238209 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:27.238313 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.238318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.239023 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.239029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.239998 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:27.240014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.240768 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.240772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:27.240856 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.240911 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 15:54:27.241811 (+ 900us) client_negotiation.cc:240] Negotiation successful
0808 15:54:27.241893 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":25,"thread_start_us":82,"threads_started":1}
I20260808 15:54:27.242393 19308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9dcfaea3384cd0b2d8c2037dcf6a7b" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d30331485054a299246450a432e7c1e" is_pre_election: true
I20260808 15:54:27.242616 19520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9dcfaea3384cd0b2d8c2037dcf6a7b" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" is_pre_election: true
I20260808 15:54:27.242697 19308 raft_consensus.cc:2454] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 0.
I20260808 15:54:27.242899 19520 raft_consensus.cc:2454] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 0.
I20260808 15:54:27.243157 18994 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629015e49462454988aa207e71601766, 8d30331485054a299246450a432e7c1e; no voters:
I20260808 15:54:27.243438 19174 raft_consensus.cc:2781] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:54:27.243552 19174 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:54:27.243613 19174 raft_consensus.cc:3037] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:27.244612 19174 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.244949 19174 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 election: Requested vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
I20260808 15:54:27.245266 19308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9dcfaea3384cd0b2d8c2037dcf6a7b" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d30331485054a299246450a432e7c1e"
I20260808 15:54:27.245364 19520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9dcfaea3384cd0b2d8c2037dcf6a7b" candidate_uuid: "629015e49462454988aa207e71601766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
I20260808 15:54:27.245383 19308 raft_consensus.cc:3037] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:27.245508 19520 raft_consensus.cc:3037] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:27.246579 19308 raft_consensus.cc:2454] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 1.
I20260808 15:54:27.246825 19520 raft_consensus.cc:2454] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629015e49462454988aa207e71601766 in term 1.
I20260808 15:54:27.246917 18994 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629015e49462454988aa207e71601766, 8d30331485054a299246450a432e7c1e; no voters:
I20260808 15:54:27.247098 19174 raft_consensus.cc:2781] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:27.247315 19174 raft_consensus.cc:686] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 LEADER]: Becoming Leader. State: Replica: 629015e49462454988aa207e71601766, State: Running, Role: LEADER
I20260808 15:54:27.247645 19174 consensus_queue.cc:237] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.250602 18880 catalog_manager.cc:5495] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 reported cstate change: term changed from 0 to 1, leader changed from <none> to 629015e49462454988aa207e71601766 (127.30.76.65). New cstate: current_term: 1 leader_uuid: "629015e49462454988aa207e71601766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } health_report { overall_health: HEALTHY } } }
W20260808 15:54:27.268680 19592 tablet.cc:2369] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:54:27.300197 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.296729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.30.76.65:42031)
0808 15:54:27.297055 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:27.297062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:27.297075 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:54:27.297152 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:27.297154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:27.297209 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:27.297313 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:54:27.297519 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.298057 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.298808 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:27.299027 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:27.299100 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:27.299332 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:54:27.299807 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:54:27.299892 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:27.300023 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 15:54:27.300072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":163,"threads_started":1}
I20260808 15:54:27.304270 19308 raft_consensus.cc:1261] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Refusing update from remote peer 629015e49462454988aa207e71601766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:27.304270 19520 raft_consensus.cc:1261] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Refusing update from remote peer 629015e49462454988aa207e71601766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:27.304984 19620 consensus_queue.cc:1035] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:27.305182 19174 consensus_queue.cc:1035] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:54:27.307559 19386 tablet.cc:2369] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:54:27.312721 19625 mvcc.cc:205] Tried to move back new op lower bound from 7316293498071298048 to 7316293497849085952. Current Snapshot: MvccSnapshot[applied={T|T < 7316293498071298048}]
I20260808 15:54:27.313001 19623 mvcc.cc:205] Tried to move back new op lower bound from 7316293498071298048 to 7316293497849085952. Current Snapshot: MvccSnapshot[applied={T|T < 7316293498071298048}]
I20260808 15:54:27.315296 19624 mvcc.cc:205] Tried to move back new op lower bound from 7316293498071298048 to 7316293497849085952. Current Snapshot: MvccSnapshot[applied={T|T < 7316293498071298048}]
I20260808 15:54:27.321995 18883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:27.322190 18883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:27.325147 18883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Sending DeleteTablet for 3 replicas of tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:27.325994 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 18939
I20260808 15:54:27.325959 19493 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:27 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:57092
I20260808 15:54:27.325986 19279 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:27 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:47450
I20260808 15:54:27.330193 19638 tablet_replica.cc:330] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: stopping tablet replica
I20260808 15:54:27.330639 19638 raft_consensus.cc:2229] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:27.331069 19638 raft_consensus.cc:2258] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:27.333431 19638 ts_tablet_manager.cc:1905] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:27.333971 19640 tablet_replica.cc:330] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: stopping tablet replica
I20260808 15:54:27.334188 19640 raft_consensus.cc:2229] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:27.334398 19640 raft_consensus.cc:2258] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:27.336854 19638 ts_tablet_manager.cc:1918] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:27.337029 19638 log.cc:1196] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:27.337397 19638 ts_tablet_manager.cc:1939] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 8d30331485054a299246450a432e7c1e: Deleting consensus metadata
I20260808 15:54:27.337767 19640 ts_tablet_manager.cc:1905] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:27.338675 18867 catalog_manager.cc:4841] TS 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b (table test-table [id=3f7f014da330487481a9c051bb607488]) successfully deleted
I20260808 15:54:27.340209 19640 ts_tablet_manager.cc:1918] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:27.340289 19640 log.cc:1196] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:27.340619 19640 ts_tablet_manager.cc:1939] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 78d8d2b6d4bb4cb8a93f05d0baa1fc22: Deleting consensus metadata
I20260808 15:54:27.341717 18868 catalog_manager.cc:4841] TS 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b (table test-table [id=3f7f014da330487481a9c051bb607488]) successfully deleted
W20260808 15:54:27.343163 18866 connection.cc:535] client connection to 127.30.76.65:42031 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 15:54:27.343328 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 19165
W20260808 15:54:27.343351 18866 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 15:54:27.348152 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.347878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42031 (local address 127.0.0.1:59024)
0808 15:54:27.348017 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.348079 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:54:27.348344 18866 catalog_manager.cc:4584] TS 629015e49462454988aa207e71601766 (127.30.76.65:42031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d9dcfaea3384cd0b2d8c2037dcf6a7b: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42031: connect: Connection refused (error 111)
I20260808 15:54:27.353583 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 19388
I20260808 15:54:27.363065 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 18836
I20260808 15:54:27.374930 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:43607
--webserver_interface=127.30.76.126
--webserver_port=34363
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:27.486213 19643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:27.486451 19643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:27.486490 19643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:27.490430 19643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:27.490491 19643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:27.490510 19643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:27.490526 19643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:27.490541 19643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:27.495816 19643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=34363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:27.496826 19643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:27.497862 19643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:27.506402 19650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:27.506470 19653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:27.506640 19643 server_base.cc:1034] running on GCE node
W20260808 15:54:27.506397 19656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:27.507608 19643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:27.508728 19643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:27.509923 19643 hybrid_clock.cc:648] HybridClock initialized: now 1786204467509906 us; error 27 us; skew 500 ppm
Aug 08 15:54:27 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204467, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:27.512719 19643 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:27.513577 19643 webserver.cc:460] Webserver started at http://127.30.76.126:34363/ using document root <none> and password file <none>
I20260808 15:54:27.514132 19643 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:27.514227 19643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:27.517866 19643 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:27.519929 19668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.521080 19643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:27.521226 19643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "81cdafedf1024c639557bd6e95fe074b"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "05671ca2f7fd17b5a610e76eabc81409"
server_key_iv: "18a4661b34dc0f647c1e39ae6671ad5e"
server_key_version: "encryptionkey@0"
I20260808 15:54:27.521752 19643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:27.531477 19643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:27.534375 19643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:27.534566 19643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:27.542692 19643 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:43607
I20260808 15:54:27.542903 19754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:43607 every 8 connection(s)
I20260808 15:54:27.543849 19643 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:27.548043 19757 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:54:27.549516 19757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap starting.
I20260808 15:54:27.551648 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 19643
I20260808 15:54:27.552053 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:42031
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:43607
--webserver_port=45099
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:27.567411 19757 log.cc:824] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:27.578543 19757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:54:27.578912 19757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b: Bootstrap complete.
I20260808 15:54:27.582978 19757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:27.583400 19757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cdafedf1024c639557bd6e95fe074b, State: Initialized, Role: FOLLOWER
I20260808 15:54:27.583911 19757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:27.584067 19757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:27.584163 19757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:27.584290 19757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 2 FOLLOWER]: Advancing to term 3
I20260808 15:54:27.586143 19757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:27.586470 19757 leader_election.cc:302] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cdafedf1024c639557bd6e95fe074b; no voters:
I20260808 15:54:27.586829 19757 leader_election.cc:288] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 15:54:27.586920 19762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 15:54:27.587106 19762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [term 3 LEADER]: Becoming Leader. State: Replica: 81cdafedf1024c639557bd6e95fe074b, State: Running, Role: LEADER
I20260808 15:54:27.587391 19762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } }
I20260808 15:54:27.588088 19757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:27.588709 19763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "81cdafedf1024c639557bd6e95fe074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } } }
I20260808 15:54:27.588835 19763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:27.589177 19764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cdafedf1024c639557bd6e95fe074b. Latest consensus state: current_term: 3 leader_uuid: "81cdafedf1024c639557bd6e95fe074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cdafedf1024c639557bd6e95fe074b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 43607 } } }
I20260808 15:54:27.589270 19764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cdafedf1024c639557bd6e95fe074b [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:27.589812 19769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:27.595073 19769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:27.595764 19769 catalog_manager.cc:1259] Loaded cluster ID: c89494ae0f564f01951ea0d6f14c6136
I20260808 15:54:27.595829 19769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 15:54:27.596961 19769 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f681cb5a9ad google::LogMessage::Fail() at ??:0
@ 0x7f681cb5d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f681cb5a4de google::LogMessage::Flush() at ??:0
@ 0x7f681cb5ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f682749dfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f682749ed2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f682749b579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f68274df640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f681edac3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f681edacc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f681edae3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f681ed9d512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f68203e16db start_thread at ??:0
@ 0x7f681b65861f clone at ??:0
W20260808 15:54:27.693501 19760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:27.693768 19760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:27.693812 19760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:27.697814 19760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:27.697867 19760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:27.697921 19760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:54:27.702988 19760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:42031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=45099
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:27.704056 19760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:27.705047 19760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:27.714620 19812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:27.714653 19814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:27.714865 19816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:27.714794 19760 server_base.cc:1034] running on GCE node
I20260808 15:54:27.715281 19760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:27.715982 19760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:27.717170 19760 hybrid_clock.cc:648] HybridClock initialized: now 1786204467717125 us; error 63 us; skew 500 ppm
Aug 08 15:54:27 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204467, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:27.720113 19760 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:54:27.721032 19760 webserver.cc:460] Webserver started at http://127.30.76.65:45099/ using document root <none> and password file <none>
I20260808 15:54:27.721594 19760 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:27.721657 19760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:27.725718 19760 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:54:27.728117 19825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.729537 19760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:54:27.729722 19760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "629015e49462454988aa207e71601766"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "14e8e0ac1575f9dc58e4e630143f97fc"
server_key_iv: "bd8c5407a2387751f6cafde06eb343da"
server_key_version: "encryptionkey@0"
I20260808 15:54:27.730273 19760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:27.752022 19760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:27.755925 19760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:27.756135 19760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:27.756776 19760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:27.757944 19837 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 15:54:27.764613 19760 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 15:54:27.764711 19760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260808 15:54:27.764819 19760 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 15:54:27.766517 19760 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 15:54:27.766593 19760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 15:54:27.766947 19837 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Bootstrap starting.
I20260808 15:54:27.777534 19760 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:42031
I20260808 15:54:27.777637 19996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:42031 every 8 connection(s)
I20260808 15:54:27.778501 19760 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:54:27.780090 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 19760
I20260808 15:54:27.780544 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:38843
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:43607
--webserver_port=36407
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:54:27.793198 19837 log.cc:824] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:27.802940 19837 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:54:27.803438 19837 tablet_bootstrap.cc:492] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Bootstrap complete.
I20260808 15:54:27.805212 19837 ts_tablet_manager.cc:1397] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Time spent bootstrapping tablet: real 0.039s user 0.027s sys 0.007s
I20260808 15:54:27.813759 19837 raft_consensus.cc:348] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.814479 19837 raft_consensus.cc:729] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629015e49462454988aa207e71601766, State: Initialized, Role: FOLLOWER
I20260808 15:54:27.815037 19837 consensus_queue.cc:260] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:27.816334 19837 ts_tablet_manager.cc:1428] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
W20260808 15:54:27.820900 19999 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:54:27.780677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:36979)
0808 15:54:27.781341 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.781404 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:27.782585 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:27.820212 (+ 37627us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":213,"threads_started":1}
W20260808 15:54:27.821403 19997 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:43607 (0 consecutive failures): Network error: Failed to ping master at 127.30.76.126:43607: Client connection negotiation failed: client connection to 127.30.76.126:43607: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 15:54:27.822160 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.821914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:40589)
0808 15:54:27.822035 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.822094 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 15:54:27.822882 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.822696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:51711)
0808 15:54:27.822794 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.822834 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:54:27.823133 19997 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:54:27.912154 20005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:27.912382 20005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:27.912426 20005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:27.916489 20005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:27.916548 20005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:27.916651 20005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:54:27.921939 20005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:38843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=36407
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:27.922914 20005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:27.923974 20005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:27.933761 20013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:27.933792 20014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:27.933832 20018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:27.934299 20005 server_base.cc:1034] running on GCE node
I20260808 15:54:27.934687 20005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:27.935243 20005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:27.936401 20005 hybrid_clock.cc:648] HybridClock initialized: now 1786204467936367 us; error 46 us; skew 500 ppm
Aug 08 15:54:27 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204467, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:27.939002 20005 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:54:27.939950 20005 webserver.cc:460] Webserver started at http://127.30.76.66:36407/ using document root <none> and password file <none>
I20260808 15:54:27.940497 20005 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:27.940589 20005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:27.944298 20005 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:27.946228 20035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.947337 20005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:27.947480 20005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "8d30331485054a299246450a432e7c1e"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "7d1ea9430f53507851f8cd987b164595"
server_key_iv: "ea35b71afca932c90df309f810d8a86b"
server_key_version: "encryptionkey@0"
I20260808 15:54:27.948052 20005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:27.959333 20005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:27.961937 20005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:27.962127 20005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:27.962666 20005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:27.963433 20005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:27.963502 20005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.963580 20005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:27.963626 20005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:27.972517 20005 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:38843
I20260808 15:54:27.972553 20206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:38843 every 8 connection(s)
I20260808 15:54:27.973446 20005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:54:27.977067 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.975346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:51783)
0808 15:54:27.975806 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.975912 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":153,"threads_started":1}
W20260808 15:54:27.977494 20208 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:43607 (0 consecutive failures): Network error: Failed to ping master at 127.30.76.126:43607: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
I20260808 15:54:27.978052 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.977856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:36767)
0808 15:54:27.977958 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.978002 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 15:54:27.978590 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:27.978416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:54921)
0808 15:54:27.978508 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:27.978547 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:54:27.978770 20208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:54:27.979626 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 20005
I20260808 15:54:27.980047 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:38447
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:43607
--webserver_port=44587
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:28.094935 20214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:28.095175 20214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:28.095216 20214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:28.099227 20214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:28.099298 20214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:28.099351 20214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:54:28.104300 20214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:38447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=44587
--tserver_master_addrs=127.30.76.126:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:28.105201 20214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:28.106196 20214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:28.115482 20221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:28.115461 20225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:28.115487 20223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:28.115718 20214 server_base.cc:1034] running on GCE node
I20260808 15:54:28.116245 20214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:28.116809 20214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:28.117998 20214 hybrid_clock.cc:648] HybridClock initialized: now 1786204468118017 us; error 64 us; skew 500 ppm
Aug 08 15:54:28 dist-test-slave-sb2z krb5kdc[18043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204468, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:28.120693 20214 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:54:28.121513 20214 webserver.cc:460] Webserver started at http://127.30.76.67:44587/ using document root <none> and password file <none>
I20260808 15:54:28.122049 20214 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:28.122164 20214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:28.125823 20214 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:28.127880 20234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:28.128974 20214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:28.129107 20214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22"
format_stamp: "Formatted at 2026-08-08 15:54:25 on dist-test-slave-sb2z"
server_key: "91710133d37fa1909da74057cb0d8b72"
server_key_iv: "5c1c123f299b1c2265bdb0342661dbbd"
server_key_version: "encryptionkey@0"
I20260808 15:54:28.129671 20214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:28.149073 20214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:28.152032 20214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:28.152189 20214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:28.152747 20214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:28.153489 20214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:28.153537 20214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:28.153576 20214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:28.153599 20214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:28.162349 20214 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:38447
I20260808 15:54:28.162390 20418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:38447 every 8 connection(s)
I20260808 15:54:28.163314 20214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:54:28.167014 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:28.165371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:60075)
0808 15:54:28.165798 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:28.165906 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260808 15:54:28.167450 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 20214
W20260808 15:54:28.167587 20420 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:43607 (0 consecutive failures): Network error: Failed to ping master at 127.30.76.126:43607: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
I20260808 15:54:28.168236 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:28.168015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:55887)
0808 15:54:28.168141 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:28.168185 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 15:54:28.168907 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:28.168704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:57549)
0808 15:54:28.168807 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:28.168851 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:54:28.169090 20420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:54:28.824193 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:28.823778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:37829)
0808 15:54:28.824020 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:28.824075 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260808 15:54:28.979746 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:28.979340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:59681)
0808 15:54:28.979605 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:28.979655 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260808 15:54:29.169916 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.169559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:52235)
0808 15:54:29.169776 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:29.169823 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260808 15:54:29.411371 20429 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:29.411607 20429 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:29.412844 20429 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
Aug 08 15:54:29 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204467, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.66@KRBTEST.COM
Aug 08 15:54:29 dist-test-slave-sb2z krb5kdc[18043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204467, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.67@KRBTEST.COM
I20260808 15:54:29.427325 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.413329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:52905 (local address 127.30.76.66:38843)
0808 15:54:29.413752 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:29.413802 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:29.413873 (+ 71us) server_negotiation.cc:407] Connection header received
0808 15:54:29.414601 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:29.414608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:29.415015 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:29.415329 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:29.416804 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.417788 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.418899 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.419636 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.419790 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:29.422639 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:29.422647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:29.422664 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:29.422704 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:29.424868 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425393 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:29.425401 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:29.425410 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:29.425522 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425876 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:29.425880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:29.425882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:29.426270 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:29.426415 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:29.426907 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 15:54:29.427166 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":76,"threads_started":1}
I20260808 15:54:29.427306 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.413025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38843 (local address 127.30.76.65:52905)
0808 15:54:29.413616 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:29.413634 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:29.413759 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:29.415378 (+ 1619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:29.415384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:29.415931 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:29.416589 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:29.416597 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.417936 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.417939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:29.418746 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:29.418756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.419759 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.419763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:29.419942 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:29.420597 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:29.420613 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:29.422360 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:29.424992 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:29.424998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:29.425011 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:29.425269 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:29.425625 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:29.425631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:29.425776 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:29.426389 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:29.426396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:29.426610 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 15:54:29.426799 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 15:54:29.427116 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":45,"thread_start_us":94,"threads_started":1}
I20260808 15:54:29.427387 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.413807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:55335 (local address 127.30.76.67:38447)
0808 15:54:29.414208 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:29.414271 (+ 63us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:29.414338 (+ 67us) server_negotiation.cc:407] Connection header received
0808 15:54:29.415417 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:29.415426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:29.415895 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:29.416193 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:29.416793 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.418668 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.419477 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.420167 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.420323 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:29.422952 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:29.422957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:29.422967 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:29.422995 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:29.425050 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425514 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:29.425519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:29.425525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:29.425645 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425952 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:29.425956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:29.425958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:29.426330 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:29.426463 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:29.426912 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 15:54:29.427193 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260808 15:54:29.427414 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.413374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:38447 (local address 127.30.76.65:55335)
0808 15:54:29.413729 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:29.413741 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:29.413830 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:29.416158 (+ 2328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:29.416161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:29.416344 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:29.416575 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:29.416597 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.418850 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.418856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:29.419356 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:29.419364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:29.420278 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:29.420282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:29.420363 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:29.420810 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:29.420824 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:29.422770 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:29.425189 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:29.425194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:29.425398 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:29.425750 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:29.425753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:29.425755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:29.425853 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:29.426450 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:29.426453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:29.426611 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 15:54:29.426798 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 15:54:29.427203 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":187,"spinlock_wait_cycles":10496,"thread_start_us":59,"threads_started":1}
W20260808 15:54:29.430035 19833 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
W20260808 15:54:29.430413 19832 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:29.430483 19832 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629015e49462454988aa207e71601766; no voters: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, 8d30331485054a299246450a432e7c1e
I20260808 15:54:29.430711 20429 raft_consensus.cc:2726] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:54:29.825162 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.824893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:35315)
0808 15:54:29.825023 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:29.825073 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 15:54:29.980790 20435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:29.980420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:45885)
0808 15:54:29.980657 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:29.980709 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":66,"threads_started":1}
I20260808 15:54:30.170945 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:30.170579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:58735)
0808 15:54:30.170811 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:30.170860 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260808 15:54:30.826326 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:30.825893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:41547)
0808 15:54:30.826171 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:30.826226 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260808 15:54:30.958127 20438 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:30.958420 20438 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:30.958832 20438 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
W20260808 15:54:30.959676 19833 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
W20260808 15:54:30.959779 19832 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:30.959887 19832 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629015e49462454988aa207e71601766; no voters: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, 8d30331485054a299246450a432e7c1e
I20260808 15:54:30.960057 20438 raft_consensus.cc:2726] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:54:30.981897 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:30.981504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:57331)
0808 15:54:30.981760 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:30.981814 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
I20260808 15:54:31.172019 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:31.171640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:48255)
0808 15:54:31.171876 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:31.171929 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":68,"threads_started":1}
I20260808 15:54:31.827626 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:31.827117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:41239)
0808 15:54:31.827375 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:31.827454 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260808 15:54:31.983095 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:31.982634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:55747)
0808 15:54:31.982952 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:31.983008 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":73,"threads_started":1}
I20260808 15:54:32.173079 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:32.172699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:50449)
0808 15:54:32.172944 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:32.172994 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":81,"threads_started":1}
I20260808 15:54:32.828989 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:32.828548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:38135)
0808 15:54:32.828841 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:32.828892 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":81,"threads_started":1}
I20260808 15:54:32.984397 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:32.983998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:43593)
0808 15:54:32.984257 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:32.984309 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260808 15:54:33.174257 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:33.173833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:47311)
0808 15:54:33.174105 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:33.174165 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
I20260808 15:54:33.271469 20456 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:33.271689 20456 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:33.272158 20456 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
W20260808 15:54:33.272886 19833 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
W20260808 15:54:33.272979 19832 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:33.273100 19832 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629015e49462454988aa207e71601766; no voters: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, 8d30331485054a299246450a432e7c1e
I20260808 15:54:33.273288 20456 raft_consensus.cc:2726] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:54:33.830159 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:33.829745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:52171)
0808 15:54:33.830007 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:33.830061 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260808 15:54:33.985446 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:33.985063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:52251)
0808 15:54:33.985312 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:33.985364 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260808 15:54:34.175390 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:34.174986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:59543)
0808 15:54:34.175243 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:34.175299 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260808 15:54:34.831476 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:34.831065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:56147)
0808 15:54:34.831324 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:34.831378 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260808 15:54:34.986569 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:34.986119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:43785)
0808 15:54:34.986423 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:34.986478 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":80,"threads_started":1}
I20260808 15:54:35.176628 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:35.176183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:37287)
0808 15:54:35.176485 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:35.176538 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":82,"threads_started":1}
I20260808 15:54:35.832754 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:35.832304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:59651)
0808 15:54:35.832592 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:35.832647 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":85,"threads_started":1}
I20260808 15:54:35.987707 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:35.987263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:44951)
0808 15:54:35.987498 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:35.987601 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":76,"threads_started":1}
I20260808 15:54:36.177973 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:36.177432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:42673)
0808 15:54:36.177818 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:36.177877 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":82,"threads_started":1}
I20260808 15:54:36.364579 20473 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:36.364720 20473 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:36.365186 20473 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
W20260808 15:54:36.365990 19832 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
W20260808 15:54:36.366107 19833 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:36.366307 19833 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629015e49462454988aa207e71601766; no voters: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, 8d30331485054a299246450a432e7c1e
I20260808 15:54:36.366477 20473 raft_consensus.cc:2726] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:54:36.833819 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:36.833408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:51457)
0808 15:54:36.833669 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:36.833722 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
I20260808 15:54:36.988783 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:36.988396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:42821)
0808 15:54:36.988640 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:36.988693 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260808 15:54:37.179195 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:37.178760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:57133)
0808 15:54:37.179031 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:37.179089 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260808 15:54:37.835139 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:37.834717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:48399)
0808 15:54:37.834972 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:37.835034 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260808 15:54:37.989959 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:37.989480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:37279)
0808 15:54:37.989814 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:37.989868 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":67,"threads_started":1}
I20260808 15:54:38.180493 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:38.180062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:51949)
0808 15:54:38.180345 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:38.180398 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
I20260808 15:54:38.836504 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:38.836004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:39075)
0808 15:54:38.836326 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:38.836385 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":137,"threads_started":1}
I20260808 15:54:38.991147 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:38.990713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:35723)
0808 15:54:38.990985 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:38.991043 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":73,"threads_started":1}
I20260808 15:54:39.181614 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:39.181248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:37617)
0808 15:54:39.181470 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:39.181522 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260808 15:54:39.837770 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:39.837348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:58147)
0808 15:54:39.837600 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:39.837669 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260808 15:54:39.992336 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:39.991920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:33673)
0808 15:54:39.992186 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:39.992241 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":69,"threads_started":1}
I20260808 15:54:40.059218 20489 raft_consensus.cc:482] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:40.059353 20489 raft_consensus.cc:504] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d30331485054a299246450a432e7c1e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38843 } } peers { permanent_uuid: "78d8d2b6d4bb4cb8a93f05d0baa1fc22" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 38447 } } peers { permanent_uuid: "629015e49462454988aa207e71601766" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42031 } }
I20260808 15:54:40.059894 20489 leader_election.cc:288] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d30331485054a299246450a432e7c1e (127.30.76.66:38843), 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447)
W20260808 15:54:40.060647 19832 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d30331485054a299246450a432e7c1e (127.30.76.66:38843): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
W20260808 15:54:40.060775 19833 leader_election.cc:341] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78d8d2b6d4bb4cb8a93f05d0baa1fc22 (127.30.76.67:38447): Not found: Tablet not found: 2d9dcfaea3384cd0b2d8c2037dcf6a7b
I20260808 15:54:40.060873 19833 leader_election.cc:302] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629015e49462454988aa207e71601766; no voters: 78d8d2b6d4bb4cb8a93f05d0baa1fc22, 8d30331485054a299246450a432e7c1e
I20260808 15:54:40.061053 20489 raft_consensus.cc:2726] T 2d9dcfaea3384cd0b2d8c2037dcf6a7b P 629015e49462454988aa207e71601766 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:54:40.182956 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:40.182410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:56031)
0808 15:54:40.182766 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:40.182816 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260808 15:54:40.838907 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:40.838506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:33439)
0808 15:54:40.838753 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:40.838808 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260808 15:54:40.993558 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:40.993166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:43843)
0808 15:54:40.993411 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:40.993466 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260808 15:54:41.184192 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:41.183806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:51713)
0808 15:54:41.184043 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:41.184095 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260808 15:54:41.840129 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:41.839734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:38805)
0808 15:54:41.839975 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:41.840029 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260808 15:54:41.994753 20498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:41.994330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:53735)
0808 15:54:41.994599 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:41.994657 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260808 15:54:42.185372 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:42.184922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:54725)
0808 15:54:42.185203 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:42.185260 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":98,"threads_started":1}
I20260808 15:54:42.841522 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:42.841121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.65:41333)
0808 15:54:42.841370 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:42.841424 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260808 15:54:42.996140 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:42.995752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.66:35911)
0808 15:54:42.995995 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:42.996049 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260808 15:54:43.170545 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 19760
I20260808 15:54:43.183358 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 20005
I20260808 15:54:43.188259 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:43.187739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:43607 (local address 127.30.76.67:54937)
0808 15:54:43.188044 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:43.188118 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":66,"threads_started":1}
I20260808 15:54:43.193032 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 20214
2026-08-08T15:54:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19431 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:54:43 dist-test-slave-sb2z krb5kdc[20509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:54:43 dist-test-slave-sb2z krb5kdc[20509](info): set up 2 sockets
Aug 08 15:54:43 dist-test-slave-sb2z krb5kdc[20509](info): commencing operation
krb5kdc: starting...
W20260808 15:54:44.410926 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:54:44 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:54:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:54:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:54:44.538030 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41345
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:44.652690 20527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:44.652912 20527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:44.652951 20527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:44.656808 20527 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 15:54:44.656853 20527 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 15:54:44.656868 20527 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 15:54:44.656883 20527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:54:44.656899 20527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:44.656917 20527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:44.656934 20527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:54:44.656948 20527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:54:44.661947 20527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:44.662834 20527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:44.663808 20527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:44.671963 20541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:44.672021 20537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:44.671993 20534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:44.672019 20527 server_base.cc:1034] running on GCE node
I20260808 15:54:44.672827 20527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:44.673957 20527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:44.675148 20527 hybrid_clock.cc:648] HybridClock initialized: now 1786204484675123 us; error 41 us; skew 500 ppm
Aug 08 15:54:44 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204484, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:44.677832 20527 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:54:44.678705 20527 webserver.cc:460] Webserver started at http://127.30.76.126:32973/ using document root <none> and password file <none>
I20260808 15:54:44.679244 20527 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:44.679337 20527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:44.679631 20527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:44.681638 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "4468cec04050480785a89bc33b52dc77"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "b078cdcc3e0281f2b7fee27cdafc01be"
server_key_iv: "c19fce698957bb79ea29b4bbe4a47032"
server_key_version: "encryptionkey@0"
I20260808 15:54:44.682199 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "4468cec04050480785a89bc33b52dc77"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "b078cdcc3e0281f2b7fee27cdafc01be"
server_key_iv: "c19fce698957bb79ea29b4bbe4a47032"
server_key_version: "encryptionkey@0"
I20260808 15:54:44.685730 20527 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:54:44.688079 20554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:44.689188 20527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:44.689328 20527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "4468cec04050480785a89bc33b52dc77"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "b078cdcc3e0281f2b7fee27cdafc01be"
server_key_iv: "c19fce698957bb79ea29b4bbe4a47032"
server_key_version: "encryptionkey@0"
I20260808 15:54:44.689435 20527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:44.717236 20527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:44.720141 20527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:44.720361 20527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:44.727548 20527 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41345
I20260808 15:54:44.727875 20636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41345 every 8 connection(s)
I20260808 15:54:44.728529 20527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:54:44.731211 20638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:44.734736 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 20527
I20260808 15:54:44.734835 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:54:44.735165 31025 external_mini_cluster.cc:1428] Setting key 9a52e7e61428abd89dd4c856f0d62b94
I20260808 15:54:44.736959 20638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap starting.
I20260808 15:54:44.739401 20638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:44.740180 20638 log.cc:824] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:44.742298 20638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: No bootstrap required, opened a new log
Aug 08 15:54:44 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:44.746059 20638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:54:44.746308 20638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:44.746393 20638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4468cec04050480785a89bc33b52dc77, State: Initialized, Role: FOLLOWER
I20260808 15:54:44.746825 20638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:54:44.746977 20638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:54:44.747121 20638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:54:44.747239 20638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:44.748400 20638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:54:44.748766 20638 leader_election.cc:302] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4468cec04050480785a89bc33b52dc77; no voters:
I20260808 15:54:44.749080 20638 leader_election.cc:288] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:54:44.749195 20643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:44.749402 20643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 LEADER]: Becoming Leader. State: Replica: 4468cec04050480785a89bc33b52dc77, State: Running, Role: LEADER
I20260808 15:54:44.749507 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:44.736569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.30.76.126:41345)
0808 15:54:44.736968 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:44.736983 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:44.737018 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:54:44.737666 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:44.737682 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:44.737985 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:44.738249 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:44.739121 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:44.740155 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:44.741247 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:44.741704 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:44.741822 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:44.744449 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:44.744455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:44.744467 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:44.744494 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:44.746258 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:44.746816 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:44.746827 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:44.746831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:44.746942 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:44.747301 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:44.747304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:44.747305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:44.747803 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:44.747957 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:44.748248 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 15:54:44.748514 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":135,"threads_started":1}
I20260808 15:54:44.749797 20643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:54:44.750332 20638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:54:44.750931 20648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4468cec04050480785a89bc33b52dc77. Latest consensus state: current_term: 1 leader_uuid: "4468cec04050480785a89bc33b52dc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } } }
I20260808 15:54:44.751072 20648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:44.752367 20645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4468cec04050480785a89bc33b52dc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } } }
I20260808 15:54:44.752471 20645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: This master's current role is: LEADER
I20260808 15:54:44.752939 20652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:54:44.755761 20652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:54:44.760941 20652 catalog_manager.cc:1347] Generated new cluster ID: facd9ae4c7014ac2b8aa479d7bbb8b92
I20260808 15:54:44.761009 20652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:54:44.784374 20652 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:54:44.785310 20652 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:54:44.796522 20652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Generated new TSK 0
I20260808 15:54:44.797325 20652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:54:44.832955 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41345
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:44.944988 20674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:44.945271 20674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:44.945349 20674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:44.949308 20674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:44.949390 20674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:44.949477 20674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:54:44.955265 20674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:44.956298 20674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:44.957315 20674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:44.966825 20681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:44.966843 20687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:44.966825 20684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:44.967144 20674 server_base.cc:1034] running on GCE node
I20260808 15:54:44.967820 20674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:44.968510 20674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:44.969679 20674 hybrid_clock.cc:648] HybridClock initialized: now 1786204484969667 us; error 28 us; skew 500 ppm
Aug 08 15:54:44 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204484, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:44.972724 20674 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:54:44.973519 20674 webserver.cc:460] Webserver started at http://127.30.76.65:39241/ using document root <none> and password file <none>
I20260808 15:54:44.974093 20674 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:44.974154 20674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:44.974314 20674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:44.976471 20674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "551621d079294eb6b86158b6a380a69e"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "ead126e6aec42e2a14ca1fdf69cedc6d"
server_key_iv: "28b3b7096741a5427c0e66301eb15a2a"
server_key_version: "encryptionkey@0"
I20260808 15:54:44.976984 20674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "551621d079294eb6b86158b6a380a69e"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "ead126e6aec42e2a14ca1fdf69cedc6d"
server_key_iv: "28b3b7096741a5427c0e66301eb15a2a"
server_key_version: "encryptionkey@0"
I20260808 15:54:44.980569 20674 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:54:44.982867 20698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:44.984133 20674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:44.984270 20674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "551621d079294eb6b86158b6a380a69e"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "ead126e6aec42e2a14ca1fdf69cedc6d"
server_key_iv: "28b3b7096741a5427c0e66301eb15a2a"
server_key_version: "encryptionkey@0"
I20260808 15:54:44.984382 20674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:44.997000 20674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:44.999686 20674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:44.999903 20674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:45.000473 20674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:45.001302 20674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:45.001376 20674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.001448 20674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:45.001500 20674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.010416 20674 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:44351
I20260808 15:54:45.010483 20866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:44351 every 8 connection(s)
I20260808 15:54:45.011329 20674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:54:45.020123 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 20674
I20260808 15:54:45.020263 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
Aug 08 15:54:45 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204484, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:45.020634 31025 external_mini_cluster.cc:1428] Setting key c0fb0ccc84ee04003ee035f543e4f647
I20260808 15:54:45.025566 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.013214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:41905 (local address 127.30.76.126:41345)
0808 15:54:45.013374 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.013377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.014154 (+ 777us) server_negotiation.cc:407] Connection header received
0808 15:54:45.015386 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.015389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.015438 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.015515 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:45.016653 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.017207 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.018271 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.018487 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.018563 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.021319 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:45.021322 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:45.021325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:45.021348 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:45.023116 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.023774 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.023780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.023785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.023890 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.024287 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.024293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.024298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.024557 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:45.024665 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.025254 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.025382 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:54:45.026321 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.013535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:41905)
0808 15:54:45.014002 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.014044 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.015189 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.015740 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.015746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.016079 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:45.016524 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.016536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.017317 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.017319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.018168 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.018177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.018614 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.018616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.018761 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.019288 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:45.019303 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:45.021142 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:45.023238 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:45.023243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:45.023254 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:45.023663 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:45.023991 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:45.023996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:45.024001 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:45.024177 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:45.024747 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:45.024757 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:45.024997 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.025190 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.025506 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260808 15:54:45.027686 20872 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41345
I20260808 15:54:45.027938 20872 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:45.028432 20872 heartbeater.cc:511] Master 127.30.76.126:41345 requested a full tablet report, sending...
I20260808 15:54:45.029981 20577 ts_manager.cc:194] Registered new tserver with Master: 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351)
I20260808 15:54:45.031131 20577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:41905
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:54:45.050609 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41345
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:45.162550 20880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:45.162833 20880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:45.162930 20880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:45.167104 20880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:45.167186 20880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:45.167273 20880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:54:45.172444 20880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:45.173465 20880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:45.174505 20880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:54:45.184271 20889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:45.184288 20885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:45.184407 20886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:45.184685 20880 server_base.cc:1034] running on GCE node
I20260808 15:54:45.185060 20880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:45.185648 20880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:45.186807 20880 hybrid_clock.cc:648] HybridClock initialized: now 1786204485186777 us; error 44 us; skew 500 ppm
Aug 08 15:54:45 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204485, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:45.193665 20880 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:54:45.194557 20880 webserver.cc:460] Webserver started at http://127.30.76.66:39453/ using document root <none> and password file <none>
I20260808 15:54:45.195113 20880 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:45.195181 20880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:45.195391 20880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:45.197422 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "3e25b5b6190c476e89109f688c0845e2"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "4ed0371fb1169e9dd705a76cd283cbf7"
server_key_iv: "30bccc92a6c77add3ec50ac95ffe130f"
server_key_version: "encryptionkey@0"
I20260808 15:54:45.197978 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "3e25b5b6190c476e89109f688c0845e2"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "4ed0371fb1169e9dd705a76cd283cbf7"
server_key_iv: "30bccc92a6c77add3ec50ac95ffe130f"
server_key_version: "encryptionkey@0"
I20260808 15:54:45.201603 20880 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 15:54:45.203918 20897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.205096 20880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:54:45.205247 20880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3e25b5b6190c476e89109f688c0845e2"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "4ed0371fb1169e9dd705a76cd283cbf7"
server_key_iv: "30bccc92a6c77add3ec50ac95ffe130f"
server_key_version: "encryptionkey@0"
I20260808 15:54:45.205358 20880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:45.216038 20880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:45.219260 20880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:45.219467 20880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:45.220088 20880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:45.220947 20880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:45.221019 20880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.221096 20880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:45.221156 20880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.230684 20880 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:43145
I20260808 15:54:45.230712 21056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:43145 every 8 connection(s)
I20260808 15:54:45.231642 20880 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:54:45.237272 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 20880
I20260808 15:54:45.237409 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:54:45.237715 31025 external_mini_cluster.cc:1428] Setting key 64fa1d359b3cb4b7fd2f8d46f8a9e1dd
Aug 08 15:54:45 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204485, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:45.246527 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.233524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:55669 (local address 127.30.76.126:41345)
0808 15:54:45.233669 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.233672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.234926 (+ 1254us) server_negotiation.cc:407] Connection header received
0808 15:54:45.235887 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.235891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.235948 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.236029 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:45.237197 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.237876 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.239246 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.239565 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.239678 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.242829 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:45.242832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:45.242834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:45.242859 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:45.244447 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.244983 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.244991 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.244992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.245072 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.245424 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.245429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.245431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.245623 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:45.245738 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.246214 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.246345 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:54:45.247278 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.233918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:55669)
0808 15:54:45.234785 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.234827 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.235684 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.236191 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.236198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.236531 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:45.237026 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.237045 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.238008 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.238012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.239110 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.239120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.239751 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.239756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.239975 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.240585 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:45.240610 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:45.242658 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:45.244580 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:45.244587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:45.244600 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:45.244861 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:45.245178 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:45.245181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:45.245183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:45.245322 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:45.245735 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:45.245741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:45.245973 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.246167 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.246472 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":541,"threads_started":1}
I20260808 15:54:45.248464 21058 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41345
I20260808 15:54:45.248733 21058 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:54:45.249298 21058 heartbeater.cc:511] Master 127.30.76.126:41345 requested a full tablet report, sending...
I20260808 15:54:45.250399 20577 ts_manager.cc:194] Registered new tserver with Master: 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145)
I20260808 15:54:45.251173 20577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:55669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:54:45.267180 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41345
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:54:45.381305 21068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:54:45.381595 21068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:54:45.381672 21068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:54:45.385650 21068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:54:45.385699 21068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:54:45.385782 21068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:54:45.390888 21068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:54:45.391909 21068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:54:45.392943 21068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:54:45.402499 21068 server_base.cc:1034] running on GCE node
W20260808 15:54:45.402416 21074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:45.402453 21073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:54:45.402416 21078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:54:45.403030 21068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:54:45.403743 21068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:54:45.404923 21068 hybrid_clock.cc:648] HybridClock initialized: now 1786204485404908 us; error 26 us; skew 500 ppm
Aug 08 15:54:45 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204485, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:54:45.407671 21068 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:54:45.408558 21068 webserver.cc:460] Webserver started at http://127.30.76.67:40915/ using document root <none> and password file <none>
I20260808 15:54:45.409145 21068 fs_manager.cc:359] Metadata directory not provided
I20260808 15:54:45.409240 21068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:54:45.409442 21068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:54:45.411371 21068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "b03f907bcbae4655be7a3675299ee5da"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "6b5db5728dccffc61a5cd52f400cf6c8"
server_key_iv: "147a71a2c475ca1de8da06ab5a0f6ae9"
server_key_version: "encryptionkey@0"
I20260808 15:54:45.411985 21068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "b03f907bcbae4655be7a3675299ee5da"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "6b5db5728dccffc61a5cd52f400cf6c8"
server_key_iv: "147a71a2c475ca1de8da06ab5a0f6ae9"
server_key_version: "encryptionkey@0"
I20260808 15:54:45.415374 21068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:54:45.417641 21086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.418740 21068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:54:45.418871 21068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "b03f907bcbae4655be7a3675299ee5da"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "6b5db5728dccffc61a5cd52f400cf6c8"
server_key_iv: "147a71a2c475ca1de8da06ab5a0f6ae9"
server_key_version: "encryptionkey@0"
I20260808 15:54:45.418977 21068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:54:45.430938 21068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:54:45.434103 21068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:54:45.434306 21068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:54:45.434876 21068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:54:45.435755 21068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:54:45.435822 21068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.435896 21068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:54:45.435940 21068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:54:45.444795 21068 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:42957
I20260808 15:54:45.444823 21260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:42957 every 8 connection(s)
I20260808 15:54:45.445674 21068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:54:45.453559 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 21068
I20260808 15:54:45.453694 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:54:45.454028 31025 external_mini_cluster.cc:1428] Setting key 41779f58a7e6d5ec3076ff056a26dce2
Aug 08 15:54:45 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204485, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:54:45.459558 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.447381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:56967 (local address 127.30.76.126:41345)
0808 15:54:45.447553 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.447575 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.448804 (+ 1229us) server_negotiation.cc:407] Connection header received
0808 15:54:45.449724 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.449728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.449790 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.449915 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:45.451173 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.451797 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.453013 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.453239 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.453369 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.456223 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:45.456227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:45.456230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:45.456282 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:45.457516 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.458027 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.458031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.458033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.458117 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.458462 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.458467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.458469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.458652 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:45.458763 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.459239 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.459361 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:54:45.460330 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.447798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:56967)
0808 15:54:45.448659 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.448700 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.449482 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.450083 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.450090 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.450474 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:54:45.450997 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.451012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.451927 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.451932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.452877 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.452887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.453435 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.453440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.453604 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.454294 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:54:45.454311 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:54:45.456054 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:54:45.457651 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:45.457657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:45.457671 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:45.457921 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:45.458219 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:54:45.458223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:54:45.458225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:54:45.458366 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:54:45.458761 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:54:45.458768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:54:45.459006 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.459196 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.459503 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":540,"threads_started":1}
I20260808 15:54:45.461369 21262 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41345
I20260808 15:54:45.461615 21262 heartbeater.cc:464] Registering TS with master...
I20260808 15:54:45.462100 21262 heartbeater.cc:511] Master 127.30.76.126:41345 requested a full tablet report, sending...
I20260808 15:54:45.463110 20577 ts_manager.cc:194] Registered new tserver with Master: b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
I20260808 15:54:45.463729 20577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:56967
I20260808 15:54:45.468036 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:54:45.477316 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.469962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33478 (local address 127.30.76.126:41345)
0808 15:54:45.470100 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.470104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.470304 (+ 200us) server_negotiation.cc:407] Connection header received
0808 15:54:45.470449 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.470453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.470502 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.470587 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:54:45.471276 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.471851 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.472706 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.472927 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.473034 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.474403 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:54:45.474407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:54:45.474410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:54:45.474441 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:54:45.475627 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.476045 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.476049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.476051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.476133 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:54:45.476413 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:54:45.476417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:54:45.476418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:54:45.476636 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:54:45.476790 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.477017 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.477128 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:54:45.480010 20577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:54:45.483045 20577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:54:45.497179 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.491487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:42957 (local address 127.0.0.1:55624)
0808 15:54:45.492151 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.492164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.492231 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.493179 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.493181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.493197 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:45.493267 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.493276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.494340 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.494347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.495623 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.495634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.496596 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.496600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.496730 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.496772 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.496908 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.497031 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":480,"threads_started":1}
I20260808 15:54:45.497192 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.491790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.30.76.67:42957)
0808 15:54:45.492531 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.492545 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.492568 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:54:45.492653 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.492657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.492852 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.493010 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:45.493368 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.494209 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.495764 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.496467 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.496594 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.496750 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.496938 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.497036 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":138,"threads_started":1}
I20260808 15:54:45.497660 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.491411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:43145 (local address 127.0.0.1:55298)
0808 15:54:45.492046 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.492096 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.492231 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.493054 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.493062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.493086 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:45.493221 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.493228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.494827 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.494831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.496004 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.496014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.497256 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.497260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.497358 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.497381 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.497457 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.497524 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":81,"threads_started":1}
I20260808 15:54:45.497745 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.491555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.30.76.66:43145)
0808 15:54:45.492531 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.492545 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.492568 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:54:45.492647 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.492652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.492852 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.493390 (+ 538us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:45.493566 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.494681 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.496142 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.496942 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.497038 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.497192 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.497499 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.497595 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":123,"threads_started":1}
I20260808 15:54:45.498487 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.492222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44351 (local address 127.0.0.1:37678)
0808 15:54:45.492460 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.492474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.492564 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.494005 (+ 1441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.494008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.494028 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:45.494113 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.494118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.495361 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.495368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.496824 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.496836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.498125 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.498128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.498193 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.498210 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.498297 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.498368 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260808 15:54:45.498832 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.492754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.30.76.65:44351)
0808 15:54:45.493536 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.493552 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.493574 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:54:45.493661 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.493667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.493850 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.494005 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:45.494246 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.495200 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.496976 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.497857 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.497964 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.498074 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.498344 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.498432 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":122,"threads_started":1}
I20260808 15:54:45.500893 20966 tablet_service.cc:1467] Processing CreateTablet for tablet ae4933a5a2be45eab490f9acf825321d (DEFAULT_TABLE table=test-table [id=709445eb765d4aaa88ce3d2d6643e3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:45.500780 21166 tablet_service.cc:1467] Processing CreateTablet for tablet ae4933a5a2be45eab490f9acf825321d (DEFAULT_TABLE table=test-table [id=709445eb765d4aaa88ce3d2d6643e3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:45.501451 20766 tablet_service.cc:1467] Processing CreateTablet for tablet ae4933a5a2be45eab490f9acf825321d (DEFAULT_TABLE table=test-table [id=709445eb765d4aaa88ce3d2d6643e3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:54:45.501853 20966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae4933a5a2be45eab490f9acf825321d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:45.501893 21166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae4933a5a2be45eab490f9acf825321d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:45.502355 20766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae4933a5a2be45eab490f9acf825321d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:54:45.508512 21287 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Bootstrap starting.
I20260808 15:54:45.509086 21290 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Bootstrap starting.
I20260808 15:54:45.510259 21291 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Bootstrap starting.
I20260808 15:54:45.511283 21290 tablet_bootstrap.cc:654] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:45.511456 21287 tablet_bootstrap.cc:654] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:45.511972 21291 tablet_bootstrap.cc:654] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Neither blocks nor log segments found. Creating new log.
I20260808 15:54:45.512400 21287 log.cc:824] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:45.512424 21290 log.cc:824] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:45.513187 21291 log.cc:824] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:54:45.514510 21290 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: No bootstrap required, opened a new log
I20260808 15:54:45.514587 21287 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: No bootstrap required, opened a new log
I20260808 15:54:45.514709 21290 ts_tablet_manager.cc:1397] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:54:45.514819 21287 ts_tablet_manager.cc:1397] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:54:45.515206 21291 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: No bootstrap required, opened a new log
I20260808 15:54:45.515373 21291 ts_tablet_manager.cc:1397] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:54:45.517689 21287 raft_consensus.cc:348] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.517946 21287 raft_consensus.cc:374] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:45.518033 21287 raft_consensus.cc:729] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e25b5b6190c476e89109f688c0845e2, State: Initialized, Role: FOLLOWER
I20260808 15:54:45.518503 21287 consensus_queue.cc:260] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.518612 21290 raft_consensus.cc:348] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.518952 21290 raft_consensus.cc:374] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:45.519047 21290 raft_consensus.cc:729] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03f907bcbae4655be7a3675299ee5da, State: Initialized, Role: FOLLOWER
I20260808 15:54:45.519181 21058 heartbeater.cc:503] Master 127.30.76.126:41345 was elected leader, sending a full tablet report...
I20260808 15:54:45.519079 21291 raft_consensus.cc:348] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.519312 21291 raft_consensus.cc:374] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:54:45.519372 21291 raft_consensus.cc:729] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551621d079294eb6b86158b6a380a69e, State: Initialized, Role: FOLLOWER
I20260808 15:54:45.519588 21287 ts_tablet_manager.cc:1428] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 15:54:45.519562 21290 consensus_queue.cc:260] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.519807 21291 consensus_queue.cc:260] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.520496 21290 ts_tablet_manager.cc:1428] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:54:45.520633 21291 ts_tablet_manager.cc:1428] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260808 15:54:45.520881 21262 heartbeater.cc:503] Master 127.30.76.126:41345 was elected leader, sending a full tablet report...
I20260808 15:54:45.520975 20872 heartbeater.cc:503] Master 127.30.76.126:41345 was elected leader, sending a full tablet report...
W20260808 15:54:45.698951 21265 tablet.cc:2369] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:54:45.732786 21060 tablet.cc:2369] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:54:45.762758 20874 tablet.cc:2369] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:54:45.859959 21303 raft_consensus.cc:482] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:54:45.860170 21303 raft_consensus.cc:504] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.861287 21303 leader_election.cc:288] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
I20260808 15:54:45.866480 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.861689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:39591 (local address 127.30.76.66:43145)
0808 15:54:45.861849 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.861855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.862001 (+ 146us) server_negotiation.cc:407] Connection header received
0808 15:54:45.862159 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.862162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.862208 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.862294 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:45.862683 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.863459 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.865203 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.865803 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.865927 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.865978 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.866286 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.866355 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 15:54:45.866480 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.861681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:47999 (local address 127.30.76.67:42957)
0808 15:54:45.861840 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.861845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.862001 (+ 156us) server_negotiation.cc:407] Connection header received
0808 15:54:45.862151 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.862155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.862208 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.862294 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:54:45.862683 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.863473 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.865203 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.865803 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.865927 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.865978 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.866286 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.866355 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:54:45.866523 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.861505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:43145 (local address 127.30.76.65:39591)
0808 15:54:45.861874 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.861889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.862034 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.862328 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.862331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.862359 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:45.862568 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.862574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.863613 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.863616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.865066 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.865072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.865934 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.865938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.866094 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.866167 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.866246 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.866339 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":177,"spinlock_wait_cycles":24320,"thread_start_us":129,"threads_started":1}
I20260808 15:54:45.866525 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.861524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:42957 (local address 127.30.76.65:47999)
0808 15:54:45.861909 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:45.861923 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:54:45.862034 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:54:45.862328 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:54:45.862331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:54:45.862359 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:54:45.862560 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.862570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.863616 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.863618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.865058 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:54:45.865070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.865934 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.865938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:54:45.866091 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.866169 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 15:54:45.866246 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:54:45.866340 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":233,"spinlock_wait_cycles":18304,"thread_start_us":104,"threads_started":1}
I20260808 15:54:45.867256 20994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae4933a5a2be45eab490f9acf825321d" candidate_uuid: "551621d079294eb6b86158b6a380a69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e25b5b6190c476e89109f688c0845e2" is_pre_election: true
I20260808 15:54:45.867471 21190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae4933a5a2be45eab490f9acf825321d" candidate_uuid: "551621d079294eb6b86158b6a380a69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b03f907bcbae4655be7a3675299ee5da" is_pre_election: true
I20260808 15:54:45.867590 20994 raft_consensus.cc:2454] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551621d079294eb6b86158b6a380a69e in term 0.
I20260808 15:54:45.867764 21190 raft_consensus.cc:2454] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551621d079294eb6b86158b6a380a69e in term 0.
I20260808 15:54:45.868121 20704 leader_election.cc:302] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e25b5b6190c476e89109f688c0845e2, 551621d079294eb6b86158b6a380a69e; no voters:
I20260808 15:54:45.868440 21303 raft_consensus.cc:2781] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:54:45.868543 21303 raft_consensus.cc:482] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:54:45.868592 21303 raft_consensus.cc:3037] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:45.869797 21303 raft_consensus.cc:504] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.870157 21303 leader_election.cc:288] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 1 election: Requested vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
I20260808 15:54:45.870474 20994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae4933a5a2be45eab490f9acf825321d" candidate_uuid: "551621d079294eb6b86158b6a380a69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e25b5b6190c476e89109f688c0845e2"
I20260808 15:54:45.870514 21190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae4933a5a2be45eab490f9acf825321d" candidate_uuid: "551621d079294eb6b86158b6a380a69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b03f907bcbae4655be7a3675299ee5da"
I20260808 15:54:45.870617 21190 raft_consensus.cc:3037] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:45.870601 20994 raft_consensus.cc:3037] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:54:45.871889 20994 raft_consensus.cc:2454] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551621d079294eb6b86158b6a380a69e in term 1.
I20260808 15:54:45.871889 21190 raft_consensus.cc:2454] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551621d079294eb6b86158b6a380a69e in term 1.
I20260808 15:54:45.872198 20704 leader_election.cc:302] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e25b5b6190c476e89109f688c0845e2, 551621d079294eb6b86158b6a380a69e; no voters:
I20260808 15:54:45.872376 21303 raft_consensus.cc:2781] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:54:45.872591 21303 raft_consensus.cc:686] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 LEADER]: Becoming Leader. State: Replica: 551621d079294eb6b86158b6a380a69e, State: Running, Role: LEADER
I20260808 15:54:45.872906 21303 consensus_queue.cc:237] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:54:45.876209 20576 catalog_manager.cc:5495] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e reported cstate change: term changed from 0 to 1, leader changed from <none> to 551621d079294eb6b86158b6a380a69e (127.30.76.65). New cstate: current_term: 1 leader_uuid: "551621d079294eb6b86158b6a380a69e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } health_report { overall_health: HEALTHY } } }
I20260808 15:54:45.949267 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:45.945823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.30.76.65:44351)
0808 15:54:45.945961 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 15:54:45.945965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:54:45.945980 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:54:45.946062 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:54:45.946064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:54:45.946123 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:54:45.946212 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:54:45.946422 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.946916 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.947786 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:54:45.948010 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:54:45.948095 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:54:45.948449 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:54:45.948870 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:54:45.948970 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:54:45.949025 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 15:54:45.949075 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:54:45.955374 21190 raft_consensus.cc:1261] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Refusing update from remote peer 551621d079294eb6b86158b6a380a69e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:45.955372 20994 raft_consensus.cc:1261] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Refusing update from remote peer 551621d079294eb6b86158b6a380a69e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:54:45.956115 21303 consensus_queue.cc:1035] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:45.956318 21308 consensus_queue.cc:1035] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:54:45.962879 21311 mvcc.cc:205] Tried to move back new op lower bound from 7316293574465884160 to 7316293574137954304. Current Snapshot: MvccSnapshot[applied={T|T < 7316293574465884160}]
I20260808 15:54:45.964986 21314 mvcc.cc:205] Tried to move back new op lower bound from 7316293574465884160 to 7316293574137954304. Current Snapshot: MvccSnapshot[applied={T|T < 7316293574465884160}]
I20260808 15:54:45.972496 20577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:45.972669 20577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:54:45.975096 20577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Sending DeleteTablet for 3 replicas of tablet ae4933a5a2be45eab490f9acf825321d
I20260808 15:54:45.975903 20966 tablet_service.cc:1514] Processing DeleteTablet for tablet ae4933a5a2be45eab490f9acf825321d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:55298
I20260808 15:54:45.976048 21166 tablet_service.cc:1514] Processing DeleteTablet for tablet ae4933a5a2be45eab490f9acf825321d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:55624
I20260808 15:54:45.976392 21322 tablet_replica.cc:330] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: stopping tablet replica
I20260808 15:54:45.976418 20766 tablet_service.cc:1514] Processing DeleteTablet for tablet ae4933a5a2be45eab490f9acf825321d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:37678
I20260808 15:54:45.976719 21322 raft_consensus.cc:2229] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:45.976871 21324 tablet_replica.cc:330] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: stopping tablet replica
I20260808 15:54:45.977042 21322 raft_consensus.cc:2258] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:45.977088 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 20674
I20260808 15:54:45.985767 21323 tablet_replica.cc:330] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: stopping tablet replica
I20260808 15:54:45.986150 21323 raft_consensus.cc:2229] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:54:45.986524 21323 raft_consensus.cc:2258] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:54:45.987668 21322 ts_tablet_manager.cc:1905] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:45.988052 21323 ts_tablet_manager.cc:1905] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:54:45.989933 21322 ts_tablet_manager.cc:1918] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:45.990056 21322 log.cc:1196] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/ae4933a5a2be45eab490f9acf825321d
I20260808 15:54:45.990363 21322 ts_tablet_manager.cc:1939] T ae4933a5a2be45eab490f9acf825321d P 3e25b5b6190c476e89109f688c0845e2: Deleting consensus metadata
I20260808 15:54:45.991647 21323 ts_tablet_manager.cc:1918] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:54:45.991762 21323 log.cc:1196] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/ae4933a5a2be45eab490f9acf825321d
I20260808 15:54:45.991744 20559 catalog_manager.cc:4841] TS 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145): tablet ae4933a5a2be45eab490f9acf825321d (table test-table [id=709445eb765d4aaa88ce3d2d6643e3f5]) successfully deleted
I20260808 15:54:45.992143 21323 ts_tablet_manager.cc:1939] T ae4933a5a2be45eab490f9acf825321d P b03f907bcbae4655be7a3675299ee5da: Deleting consensus metadata
I20260808 15:54:45.993500 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 20880
I20260808 15:54:45.993602 20556 catalog_manager.cc:4841] TS b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957): tablet ae4933a5a2be45eab490f9acf825321d (table test-table [id=709445eb765d4aaa88ce3d2d6643e3f5]) successfully deleted
W20260808 15:54:45.993847 20556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:44351 (error 108)
I20260808 15:54:46.004951 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:54:46.004265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44351 (local address 127.0.0.1:37694)
0808 15:54:46.004722 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 15:54:46.004819 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":68,"threads_started":1}
W20260808 15:54:46.005128 20556 catalog_manager.cc:4584] TS 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae4933a5a2be45eab490f9acf825321d: Network error: Client connection negotiation failed: client connection to 127.30.76.65:44351: connect: Connection refused (error 111)
I20260808 15:54:46.008181 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 21068
I20260808 15:54:46.017858 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 20527
I20260808 15:55:06.029776 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41345
--webserver_interface=127.30.76.126
--webserver_port=32973
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:06.142911 21329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:06.143179 21329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:06.143217 21329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:06.147231 21329 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 15:55:06.147287 21329 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 15:55:06.147303 21329 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 15:55:06.147317 21329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:06.147379 21329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:06.147410 21329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:06.147441 21329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:06.147501 21329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:06.152732 21329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=32973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:06.153762 21329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:06.154788 21329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:06.168666 21341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:06.168591 21337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:06.168612 21329 server_base.cc:1034] running on GCE node
W20260808 15:55:06.168562 21339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:06.169535 21329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:06.170642 21329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:06.171799 21329 hybrid_clock.cc:648] HybridClock initialized: now 1786204506171767 us; error 44 us; skew 500 ppm
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:06.182200 21329 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:55:06.183185 21329 webserver.cc:460] Webserver started at http://127.30.76.126:32973/ using document root <none> and password file <none>
I20260808 15:55:06.183837 21329 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:06.183940 21329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:06.187839 21329 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 15:55:06.190021 21351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.191226 21329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:06.191371 21329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "4468cec04050480785a89bc33b52dc77"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "b078cdcc3e0281f2b7fee27cdafc01be"
server_key_iv: "c19fce698957bb79ea29b4bbe4a47032"
server_key_version: "encryptionkey@0"
I20260808 15:55:06.192061 21329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:06.201436 21329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:06.204097 21329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:06.204295 21329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:06.211509 21329 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41345
I20260808 15:55:06.211853 21426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41345 every 8 connection(s)
I20260808 15:55:06.212656 21329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:06.217278 21428 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:55:06.218132 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 21329
I20260808 15:55:06.218681 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:44351
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:41345
--webserver_port=39241
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:55:06.219396 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap starting.
I20260808 15:55:06.237923 21428 log.cc:824] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:06.246573 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:55:06.246951 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap complete.
I20260808 15:55:06.251010 21428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:06.251410 21428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4468cec04050480785a89bc33b52dc77, State: Initialized, Role: FOLLOWER
I20260808 15:55:06.251948 21428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:06.252096 21428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:06.252187 21428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:06.252301 21428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:55:06.254192 21428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:06.254515 21428 leader_election.cc:302] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4468cec04050480785a89bc33b52dc77; no voters:
I20260808 15:55:06.254828 21428 leader_election.cc:288] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:55:06.254987 21436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:55:06.255316 21436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 LEADER]: Becoming Leader. State: Replica: 4468cec04050480785a89bc33b52dc77, State: Running, Role: LEADER
I20260808 15:55:06.255697 21436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:06.255999 21428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:06.256705 21436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4468cec04050480785a89bc33b52dc77. Latest consensus state: current_term: 2 leader_uuid: "4468cec04050480785a89bc33b52dc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } } }
I20260808 15:55:06.256824 21436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:06.257297 21437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4468cec04050480785a89bc33b52dc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } } }
I20260808 15:55:06.257458 21437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:06.261144 21460 catalog_manager.cc:1259] Loaded cluster ID: facd9ae4c7014ac2b8aa479d7bbb8b92
I20260808 15:55:06.261204 21460 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: loading cluster ID for follower catalog manager: success
I20260808 15:55:06.263656 21460 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: acquiring CA information for follower catalog manager: success
I20260808 15:55:06.265211 21460 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:55:06.265345 21446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:06.267740 21446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:06.268239 21446 catalog_manager.cc:1259] Loaded cluster ID: facd9ae4c7014ac2b8aa479d7bbb8b92
I20260808 15:55:06.268301 21446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:06.269116 21446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:06.269615 21446 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Loaded TSK: 0
I20260808 15:55:06.294502 21446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Generated new TSK 1
I20260808 15:55:06.294750 21446 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:55:06.334017 21431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:06.334288 21431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:06.334381 21431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:06.338578 21431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:06.338637 21431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:06.338724 21431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:06.343858 21431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=39241
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:06.344812 21431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:06.345849 21431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:06.356194 21471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:06.356194 21472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:06.356503 21431 server_base.cc:1034] running on GCE node
W20260808 15:55:06.356735 21475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:06.357216 21431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:06.357837 21431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:06.359069 21431 hybrid_clock.cc:648] HybridClock initialized: now 1786204506359039 us; error 49 us; skew 500 ppm
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:06.361888 21431 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:55:06.362768 21431 webserver.cc:460] Webserver started at http://127.30.76.65:39241/ using document root <none> and password file <none>
I20260808 15:55:06.363317 21431 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:06.363391 21431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:06.367579 21431 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:55:06.369915 21485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.371184 21431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:55:06.371320 21431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "551621d079294eb6b86158b6a380a69e"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "ead126e6aec42e2a14ca1fdf69cedc6d"
server_key_iv: "28b3b7096741a5427c0e66301eb15a2a"
server_key_version: "encryptionkey@0"
I20260808 15:55:06.371910 21431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:06.385838 21431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:06.389119 21431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:06.389328 21431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:06.389916 21431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:06.391088 21494 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 15:55:06.396283 21431 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 15:55:06.396356 21431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 15:55:06.396440 21431 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 15:55:06.398047 21431 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 15:55:06.398115 21431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 15:55:06.398408 21494 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Bootstrap starting.
I20260808 15:55:06.409001 21431 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:44351
I20260808 15:55:06.409170 21671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:44351 every 8 connection(s)
I20260808 15:55:06.410029 21431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:06.416479 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 21431
I20260808 15:55:06.416983 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:43145
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:41345
--webserver_port=39453
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:55:06.420245 21494 log.cc:824] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:06.428349 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.411980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:37925 (local address 127.30.76.126:41345)
0808 15:55:06.412465 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.412485 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.412951 (+ 466us) server_negotiation.cc:407] Connection header received
0808 15:55:06.414041 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.414051 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.414320 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.414633 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:06.415777 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.417234 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.419047 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.419833 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.419982 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.423270 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:06.423277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:06.423303 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:06.423331 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:06.425145 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.425686 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.425692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.425695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.425823 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.426184 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.426188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.426190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.426517 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:06.426676 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.427209 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.427470 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":111,"threads_started":1}
I20260808 15:55:06.428361 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.412022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:37925)
0808 15:55:06.412762 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.412825 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.413633 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.414700 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.414708 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.415120 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:06.415618 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.415633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.417422 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.417428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.418869 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.418885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.420252 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.420257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.420483 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.421560 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:06.421577 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:06.423072 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:06.425271 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.425277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.425290 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.425561 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.425923 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.425927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.425929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.426089 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.426661 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:06.426667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:06.426900 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.427089 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.427420 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":146,"threads_started":1}
I20260808 15:55:06.429991 21494 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:55:06.430485 21494 tablet_bootstrap.cc:492] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Bootstrap complete.
I20260808 15:55:06.431121 21674 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41345
I20260808 15:55:06.431363 21674 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:06.431902 21674 heartbeater.cc:511] Master 127.30.76.126:41345 requested a full tablet report, sending...
I20260808 15:55:06.432031 21494 ts_tablet_manager.cc:1397] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.007s
I20260808 15:55:06.434552 21375 ts_manager.cc:194] Registered new tserver with Master: 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351)
I20260808 15:55:06.434916 21494 raft_consensus.cc:348] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.435420 21494 raft_consensus.cc:729] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551621d079294eb6b86158b6a380a69e, State: Initialized, Role: FOLLOWER
I20260808 15:55:06.435947 21494 consensus_queue.cc:260] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.436902 21494 ts_tablet_manager.cc:1428] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:55:06.438295 21375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:37925
I20260808 15:55:06.440863 21674 heartbeater.cc:503] Master 127.30.76.126:41345 was elected leader, sending a full tablet report...
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.65@KRBTEST.COM
I20260808 15:55:06.448578 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.437605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44351 (local address 127.0.0.1:54958)
0808 15:55:06.438301 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.438346 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.438553 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.439103 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.439109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.439617 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:06.439793 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.439805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.441062 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.441065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.442049 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.442061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.442828 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.442831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.442910 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.443448 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:06.443460 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:06.444878 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:06.446955 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.446963 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.446967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.447190 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.447500 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.447502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.447504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.447630 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.448113 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:06.448118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:06.448212 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.448314 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.448449 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":112,"threads_started":1}
I20260808 15:55:06.448750 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.437697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.30.76.65:44351)
0808 15:55:06.438032 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.438043 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.438453 (+ 410us) server_negotiation.cc:407] Connection header received
0808 15:55:06.438721 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.438727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.438944 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.439093 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:06.439933 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.440948 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.442202 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.442711 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.442817 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.445018 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:06.445027 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:06.445040 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:06.445067 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:06.446837 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.447302 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.447307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.447309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.447405 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.447718 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.447721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.447723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.448001 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:06.448131 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.448438 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.448598 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":96,"threads_started":1}
I20260808 15:55:06.449631 21557 tablet_service.cc:1514] Processing DeleteTablet for tablet ae4933a5a2be45eab490f9acf825321d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:54:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:54958
I20260808 15:55:06.450434 21685 tablet_replica.cc:330] stopping tablet replica
I20260808 15:55:06.450742 21685 raft_consensus.cc:2229] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:06.450992 21685 raft_consensus.cc:2258] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:06.452108 21685 ts_tablet_manager.cc:1905] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:06.456521 21685 ts_tablet_manager.cc:1918] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:06.456622 21685 log.cc:1196] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal/wals/ae4933a5a2be45eab490f9acf825321d
I20260808 15:55:06.456902 21685 ts_tablet_manager.cc:1939] T ae4933a5a2be45eab490f9acf825321d P 551621d079294eb6b86158b6a380a69e: Deleting consensus metadata
I20260808 15:55:06.458788 21352 catalog_manager.cc:4841] TS 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351): tablet ae4933a5a2be45eab490f9acf825321d (table test-table [id=709445eb765d4aaa88ce3d2d6643e3f5]) successfully deleted
W20260808 15:55:06.543483 21679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:06.543799 21679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:06.543843 21679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:06.547812 21679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:06.547865 21679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:06.547920 21679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:06.552901 21679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:43145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=39453
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:06.553803 21679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:06.554728 21679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:06.564096 21696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:06.564112 21701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:06.564106 21695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:06.564142 21679 server_base.cc:1034] running on GCE node
I20260808 15:55:06.564726 21679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:06.565348 21679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:06.566514 21679 hybrid_clock.cc:648] HybridClock initialized: now 1786204506566482 us; error 47 us; skew 500 ppm
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:06.569306 21679 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:55:06.570169 21679 webserver.cc:460] Webserver started at http://127.30.76.66:39453/ using document root <none> and password file <none>
I20260808 15:55:06.570711 21679 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:06.570789 21679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:06.574393 21679 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:06.576442 21711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.577594 21679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:06.577723 21679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3e25b5b6190c476e89109f688c0845e2"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "4ed0371fb1169e9dd705a76cd283cbf7"
server_key_iv: "30bccc92a6c77add3ec50ac95ffe130f"
server_key_version: "encryptionkey@0"
I20260808 15:55:06.578217 21679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:06.588220 21679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:06.590811 21679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:06.591002 21679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:06.591621 21679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:06.592415 21679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:06.592463 21679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.592533 21679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:06.592574 21679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.602161 21679 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:43145
I20260808 15:55:06.602195 21885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:43145 every 8 connection(s)
I20260808 15:55:06.603197 21679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:06.607230 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 21679
I20260808 15:55:06.607754 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:42957
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:41345
--webserver_port=40915
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:06.617858 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.605028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:39165 (local address 127.30.76.126:41345)
0808 15:55:06.605164 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.605168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.605982 (+ 814us) server_negotiation.cc:407] Connection header received
0808 15:55:06.607325 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.607328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.607396 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.607498 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:06.608767 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.609365 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.610554 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.610962 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.611088 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.613566 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:06.613572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:06.613578 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:06.613614 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:06.615533 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.616049 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.616055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.616058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.616169 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.616499 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.616505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.616509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.616771 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:06.616885 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.617460 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.617617 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:55:06.618422 21889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.605404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:39165)
0808 15:55:06.605830 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.605871 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.607106 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.607700 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.607707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.608136 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:06.608602 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.608615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.609533 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.609538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.610413 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.610424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.611118 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.611121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.611260 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.611897 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:06.611913 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:06.613387 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:06.615675 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.615679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.615690 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.615936 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.616272 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.616274 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.616275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.616401 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.616909 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:06.616917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:06.617211 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.617365 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.617594 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":116,"threads_started":1}
I20260808 15:55:06.619591 21886 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41345
I20260808 15:55:06.619848 21886 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:06.620316 21886 heartbeater.cc:511] Master 127.30.76.126:41345 requested a full tablet report, sending...
I20260808 15:55:06.621340 21375 ts_manager.cc:194] Registered new tserver with Master: 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145)
I20260808 15:55:06.621896 21375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:39165
W20260808 15:55:06.723920 21890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:06.724254 21890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:06.724339 21890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:06.728264 21890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:06.728351 21890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:06.728444 21890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:06.733597 21890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:42957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=40915
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:06.734820 21890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:06.735890 21890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:06.745306 21897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:06.745318 21899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:06.745724 21902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:06.746024 21890 server_base.cc:1034] running on GCE node
I20260808 15:55:06.746428 21890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:06.747081 21890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:06.748277 21890 hybrid_clock.cc:648] HybridClock initialized: now 1786204506748269 us; error 32 us; skew 500 ppm
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:06.750926 21890 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:55:06.751981 21890 webserver.cc:460] Webserver started at http://127.30.76.67:40915/ using document root <none> and password file <none>
I20260808 15:55:06.752559 21890 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:06.752666 21890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:06.756529 21890 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:55:06.758750 21911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.759967 21890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:06.760103 21890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "b03f907bcbae4655be7a3675299ee5da"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "6b5db5728dccffc61a5cd52f400cf6c8"
server_key_iv: "147a71a2c475ca1de8da06ab5a0f6ae9"
server_key_version: "encryptionkey@0"
I20260808 15:55:06.760772 21890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:06.778172 21890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:06.781059 21890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:06.781262 21890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:06.781874 21890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:06.782718 21890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:06.782788 21890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.782860 21890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:06.782913 21890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:06.792022 21890 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:42957
I20260808 15:55:06.792112 22079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:42957 every 8 connection(s)
I20260808 15:55:06.793138 21890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:06.795512 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 21890
Aug 08 15:55:06 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:06.805546 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.796336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.30.76.126:41345)
0808 15:55:06.796581 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.796586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.796791 (+ 205us) server_negotiation.cc:407] Connection header received
0808 15:55:06.796891 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.796894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.796956 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.797107 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:06.797843 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.798748 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.799694 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.800139 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.800272 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.801732 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:06.801736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:06.801739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:06.801767 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:06.803678 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.804133 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.804137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.804139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.804246 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.804525 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.804531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.804536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.804798 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:06.804958 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.805189 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.805296 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":70,"threads_started":1}
I20260808 15:55:06.809597 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.795262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:38513 (local address 127.30.76.126:41345)
0808 15:55:06.795429 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.795435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.797059 (+ 1624us) server_negotiation.cc:407] Connection header received
0808 15:55:06.798717 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.798719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.798757 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.798846 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:06.800294 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.800986 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.802536 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.802862 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.802939 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.806006 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:06.806009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:06.806011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:06.806032 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:06.807652 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.808141 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.808145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.808146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.808226 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.808568 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.808572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.808573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.808747 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:06.808852 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.809332 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.809442 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:55:06.810362 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.795754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:38513)
0808 15:55:06.796885 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.796943 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.798472 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.799014 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.799021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.799384 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:06.800139 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.800160 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.801120 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.801123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.802405 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.802419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.803026 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.803032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.803257 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.803919 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:06.803936 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:06.805845 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:06.807763 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.807768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.807782 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.808032 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.808325 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:06.808328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:06.808329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:06.808466 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:06.808852 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:06.808860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:06.809091 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.809273 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.809574 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"thread_start_us":147,"threads_started":1}
I20260808 15:55:06.811411 22080 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41345
I20260808 15:55:06.811710 22080 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:06.812223 22080 heartbeater.cc:511] Master 127.30.76.126:41345 requested a full tablet report, sending...
I20260808 15:55:06.813273 21375 ts_manager.cc:194] Registered new tserver with Master: b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
I20260808 15:55:06.813868 21375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:38513
I20260808 15:55:06.819108 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:06.823334 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.819745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.30.76.126:41345)
0808 15:55:06.819879 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.819884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.820107 (+ 223us) server_negotiation.cc:407] Connection header received
0808 15:55:06.820305 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.820308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.820408 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.820516 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:06.820773 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.821278 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.822144 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.822374 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.822484 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.822808 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:06.822932 (+ 124us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 15:55:06.823201 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55404: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 15:55:06.823431 21677 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55404: token expired
W20260808 15:55:06.823840 21271 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.76.126:41345)
I20260808 15:55:06.831144 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.824294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.30.76.126:41345)
0808 15:55:06.824418 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.824422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.824436 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:55:06.824517 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.824520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.824558 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.824664 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:06.825262 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.825745 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.826536 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.826759 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.826865 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.828227 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:06.828230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:06.828233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:06.828264 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:06.829513 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.829943 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.829947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.829948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.830031 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:06.830317 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:06.830321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:06.830323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:06.830494 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:06.830634 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.830867 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.830984 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:55:06.833711 21375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:06.837035 21375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:06.845177 21557 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba2edcde4fd47999ad62e4387b7ae5a (DEFAULT_TABLE table=test-table [id=c0baf154de8f44b98cf54cfadd5bb50c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:06.845875 21557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba2edcde4fd47999ad62e4387b7ae5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:06.848610 21494 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Bootstrap starting.
I20260808 15:55:06.849074 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.843862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:43145 (local address 127.0.0.1:34110)
0808 15:55:06.843983 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.843998 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.844088 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.844955 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.844959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.844982 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:06.845088 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.845094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.846553 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.846556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.847666 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.847676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.848604 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.848610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.848710 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.848765 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.848824 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.848911 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":8}
I20260808 15:55:06.849484 21494 tablet_bootstrap.cc:654] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:06.849573 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.844058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.30.76.66:43145)
0808 15:55:06.844427 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.844439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.844465 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:55:06.844554 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.844562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.844786 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.844960 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:06.845191 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.846421 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.847794 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.848478 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.849224 (+ 746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.849341 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.849393 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.849465 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":152,"threads_started":1}
I20260808 15:55:06.850857 21494 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: No bootstrap required, opened a new log
I20260808 15:55:06.850970 21494 ts_tablet_manager.cc:1397] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 15:55:06.851410 21494 raft_consensus.cc:348] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.851516 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.844182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:42957 (local address 127.0.0.1:39902)
0808 15:55:06.844524 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.844539 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.844646 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.845667 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.845672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.845693 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:06.845787 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.845795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.847121 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.847124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.848285 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.848298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.851113 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.851117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.851207 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.851231 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.851316 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.851392 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":105,"threads_started":1}
I20260808 15:55:06.851552 21494 raft_consensus.cc:374] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:06.851588 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.844457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.30.76.67:42957)
0808 15:55:06.844962 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.844974 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.844996 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:55:06.845069 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.845074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.845279 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.845444 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:06.845918 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.846990 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.848437 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.850958 (+ 2521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.851080 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.851229 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.851344 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.851429 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":179,"threads_started":1}
I20260808 15:55:06.851596 21494 raft_consensus.cc:729] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551621d079294eb6b86158b6a380a69e, State: Initialized, Role: FOLLOWER
I20260808 15:55:06.851718 21494 consensus_queue.cc:260] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.852443 21494 ts_tablet_manager.cc:1428] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 15:55:06.852734 21782 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba2edcde4fd47999ad62e4387b7ae5a (DEFAULT_TABLE table=test-table [id=c0baf154de8f44b98cf54cfadd5bb50c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:06.853914 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba2edcde4fd47999ad62e4387b7ae5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:06.854079 21984 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba2edcde4fd47999ad62e4387b7ae5a (DEFAULT_TABLE table=test-table [id=c0baf154de8f44b98cf54cfadd5bb50c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:06.855115 21984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba2edcde4fd47999ad62e4387b7ae5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:06.861080 22096 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Bootstrap starting.
I20260808 15:55:06.861444 22097 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Bootstrap starting.
I20260808 15:55:06.863719 22096 tablet_bootstrap.cc:654] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:06.863870 22097 tablet_bootstrap.cc:654] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:06.864533 22096 log.cc:824] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:06.864641 22097 log.cc:824] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:06.866324 22096 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: No bootstrap required, opened a new log
I20260808 15:55:06.866338 22097 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: No bootstrap required, opened a new log
I20260808 15:55:06.866525 22096 ts_tablet_manager.cc:1397] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260808 15:55:06.866545 22097 ts_tablet_manager.cc:1397] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 15:55:06.869640 22096 raft_consensus.cc:348] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.869680 22097 raft_consensus.cc:348] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.869867 22096 raft_consensus.cc:374] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:06.869863 22097 raft_consensus.cc:374] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:06.869947 22096 raft_consensus.cc:729] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e25b5b6190c476e89109f688c0845e2, State: Initialized, Role: FOLLOWER
I20260808 15:55:06.869951 22097 raft_consensus.cc:729] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03f907bcbae4655be7a3675299ee5da, State: Initialized, Role: FOLLOWER
I20260808 15:55:06.870373 22096 consensus_queue.cc:260] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.870373 22097 consensus_queue.cc:260] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.870954 21886 heartbeater.cc:503] Master 127.30.76.126:41345 was elected leader, sending a full tablet report...
I20260808 15:55:06.870963 22080 heartbeater.cc:503] Master 127.30.76.126:41345 was elected leader, sending a full tablet report...
I20260808 15:55:06.871304 22096 ts_tablet_manager.cc:1428] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 15:55:06.871304 22097 ts_tablet_manager.cc:1428] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 15:55:06.882586 22102 raft_consensus.cc:482] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:06.882746 22102 raft_consensus.cc:504] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.883831 22102 leader_election.cc:288] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351)
I20260808 15:55:06.887444 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.883993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:43145 (local address 127.30.76.67:45331)
0808 15:55:06.884160 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.884176 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.884260 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.884746 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.884749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.884766 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:06.884881 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.884886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.885604 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.885611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.886382 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.886391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.887017 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.887020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.887133 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.887162 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.887248 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.887318 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":1}
I20260808 15:55:06.887444 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.884428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:45331 (local address 127.30.76.66:43145)
0808 15:55:06.884558 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.884561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.884572 (+ 11us) server_negotiation.cc:407] Connection header received
0808 15:55:06.884609 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.884611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.884646 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.884736 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:06.884991 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.885495 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.886507 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.886897 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.887017 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.887069 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.887266 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.887329 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:55:06.887658 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.884238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:42931 (local address 127.30.76.65:44351)
0808 15:55:06.884366 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.884371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.884532 (+ 161us) server_negotiation.cc:407] Connection header received
0808 15:55:06.884716 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.884723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.884773 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.884869 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:06.885128 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.885662 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.886698 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.887126 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.887265 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.887408 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.887459 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.887524 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:55:06.887676 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.884055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44351 (local address 127.30.76.67:42931)
0808 15:55:06.884445 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.884460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.884603 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:06.884879 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:06.884882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:06.884899 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:06.885015 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.885020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.885771 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.885776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.886570 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:06.886579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.887256 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.887259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:06.887354 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.887375 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:55:06.887437 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 15:55:06.887505 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":35,"thread_start_us":116,"threads_started":1}
I20260808 15:55:06.888181 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba2edcde4fd47999ad62e4387b7ae5a" candidate_uuid: "b03f907bcbae4655be7a3675299ee5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e25b5b6190c476e89109f688c0845e2" is_pre_election: true
I20260808 15:55:06.888183 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba2edcde4fd47999ad62e4387b7ae5a" candidate_uuid: "b03f907bcbae4655be7a3675299ee5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551621d079294eb6b86158b6a380a69e" is_pre_election: true
I20260808 15:55:06.888437 21589 raft_consensus.cc:2454] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03f907bcbae4655be7a3675299ee5da in term 0.
I20260808 15:55:06.888440 21816 raft_consensus.cc:2454] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03f907bcbae4655be7a3675299ee5da in term 0.
I20260808 15:55:06.888935 21915 leader_election.cc:302] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3e25b5b6190c476e89109f688c0845e2, 551621d079294eb6b86158b6a380a69e, b03f907bcbae4655be7a3675299ee5da; no voters:
I20260808 15:55:06.889206 22102 raft_consensus.cc:2781] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:06.889292 22102 raft_consensus.cc:482] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:55:06.889336 22102 raft_consensus.cc:3037] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:06.890261 22102 raft_consensus.cc:504] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.890560 22102 leader_election.cc:288] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [CANDIDATE]: Term 1 election: Requested vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351)
I20260808 15:55:06.890829 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba2edcde4fd47999ad62e4387b7ae5a" candidate_uuid: "b03f907bcbae4655be7a3675299ee5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e25b5b6190c476e89109f688c0845e2"
I20260808 15:55:06.890911 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba2edcde4fd47999ad62e4387b7ae5a" candidate_uuid: "b03f907bcbae4655be7a3675299ee5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551621d079294eb6b86158b6a380a69e"
I20260808 15:55:06.890956 21816 raft_consensus.cc:3037] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:06.891073 21589 raft_consensus.cc:3037] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:06.892066 21816 raft_consensus.cc:2454] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03f907bcbae4655be7a3675299ee5da in term 1.
I20260808 15:55:06.892163 21589 raft_consensus.cc:2454] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03f907bcbae4655be7a3675299ee5da in term 1.
I20260808 15:55:06.892395 21918 leader_election.cc:302] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e25b5b6190c476e89109f688c0845e2, b03f907bcbae4655be7a3675299ee5da; no voters:
I20260808 15:55:06.892638 22102 raft_consensus.cc:2781] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:06.892846 22102 raft_consensus.cc:686] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 1 LEADER]: Becoming Leader. State: Replica: b03f907bcbae4655be7a3675299ee5da, State: Running, Role: LEADER
I20260808 15:55:06.893148 22102 consensus_queue.cc:237] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:06.896137 21375 catalog_manager.cc:5495] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da reported cstate change: term changed from 0 to 1, leader changed from <none> to b03f907bcbae4655be7a3675299ee5da (127.30.76.67). New cstate: current_term: 1 leader_uuid: "b03f907bcbae4655be7a3675299ee5da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } health_report { overall_health: UNKNOWN } } }
W20260808 15:55:06.912159 21676 tablet.cc:2369] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:06.915329 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:06.912198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.30.76.67:42957)
0808 15:55:06.912330 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:06.912335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:06.912352 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:55:06.912505 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:06.912508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:06.912573 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:06.912694 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:06.912884 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.913363 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.914146 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:06.914339 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:06.914412 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:06.914654 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:06.915019 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:06.915106 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:06.915167 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 15:55:06.915212 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:55:06.921452 21589 raft_consensus.cc:1261] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Refusing update from remote peer b03f907bcbae4655be7a3675299ee5da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:06.921458 21816 raft_consensus.cc:1261] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Refusing update from remote peer b03f907bcbae4655be7a3675299ee5da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:06.922176 22106 consensus_queue.cc:1035] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [LEADER]: Connected to new peer: Peer: permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:06.922359 22102 consensus_queue.cc:1035] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:06.928309 22110 mvcc.cc:205] Tried to move back new op lower bound from 7316293660343476224 to 7316293660236931072. Current Snapshot: MvccSnapshot[applied={T|T < 7316293660343476224}]
I20260808 15:55:06.937668 21375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:06.937814 21375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:06.940030 21375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Sending DeleteTablet for 3 replicas of tablet 5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:06.940711 21782 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba2edcde4fd47999ad62e4387b7ae5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:06 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:34110
I20260808 15:55:06.940747 21557 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba2edcde4fd47999ad62e4387b7ae5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:06 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:54958
I20260808 15:55:06.940781 21984 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba2edcde4fd47999ad62e4387b7ae5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:06 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:39902
I20260808 15:55:06.940945 21685 tablet_replica.cc:330] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: stopping tablet replica
I20260808 15:55:06.941075 21685 raft_consensus.cc:2229] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:06.941213 21685 raft_consensus.cc:2258] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:06.941258 22122 tablet_replica.cc:330] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: stopping tablet replica
I20260808 15:55:06.941550 22122 raft_consensus.cc:2229] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:06.941679 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 21431
I20260808 15:55:06.941920 22122 raft_consensus.cc:2258] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:06.948462 22122 ts_tablet_manager.cc:1905] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:06.951507 22122 ts_tablet_manager.cc:1918] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:06.951719 22122 log.cc:1196] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:06.951889 22125 tablet_replica.cc:330] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: stopping tablet replica
I20260808 15:55:06.952215 22122 ts_tablet_manager.cc:1939] T 5ba2edcde4fd47999ad62e4387b7ae5a P 3e25b5b6190c476e89109f688c0845e2: Deleting consensus metadata
I20260808 15:55:06.952358 22125 raft_consensus.cc:2229] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:55:06.952812 22125 raft_consensus.cc:2258] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:06.953831 21355 catalog_manager.cc:4841] TS 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145): tablet 5ba2edcde4fd47999ad62e4387b7ae5a (table test-table [id=c0baf154de8f44b98cf54cfadd5bb50c]) successfully deleted
I20260808 15:55:06.954471 22125 ts_tablet_manager.cc:1905] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:06.957934 22125 ts_tablet_manager.cc:1918] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:06.958056 22125 log.cc:1196] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:06.958464 22125 ts_tablet_manager.cc:1939] T 5ba2edcde4fd47999ad62e4387b7ae5a P b03f907bcbae4655be7a3675299ee5da: Deleting consensus metadata
W20260808 15:55:06.959267 21352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:44351 (error 108)
I20260808 15:55:06.959829 21352 catalog_manager.cc:4841] TS b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957): tablet 5ba2edcde4fd47999ad62e4387b7ae5a (table test-table [id=c0baf154de8f44b98cf54cfadd5bb50c]) successfully deleted
W20260808 15:55:06.960510 22092 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:55:06.960154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44351 (local address 127.0.0.1:54968)
0808 15:55:06.960302 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:06.960320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:06.960400 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:44351: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 15:55:06.960650 21352 catalog_manager.cc:4584] TS 551621d079294eb6b86158b6a380a69e (127.30.76.65:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ba2edcde4fd47999ad62e4387b7ae5a: Network error: Client connection negotiation failed: client connection to 127.30.76.65:44351: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 15:55:06.961544 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 21679
I20260808 15:55:06.973984 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 21890
I20260808 15:55:06.987021 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 21329
I20260808 15:55:27.002691 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41345
--webserver_interface=127.30.76.126
--webserver_port=32973
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:27.117933 22128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:27.118218 22128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:27.118294 22128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:27.122324 22128 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 15:55:27.122403 22128 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 15:55:27.122427 22128 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 15:55:27.122450 22128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:27.122493 22128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:27.122529 22128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:27.122586 22128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:27.122627 22128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:27.127787 22128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=32973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:27.128759 22128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:27.129787 22128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:27.138411 22138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:27.138438 22137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:27.138438 22142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:27.138594 22128 server_base.cc:1034] running on GCE node
I20260808 15:55:27.139377 22128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:27.140480 22128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:27.141675 22128 hybrid_clock.cc:648] HybridClock initialized: now 1786204527141663 us; error 26 us; skew 500 ppm
Aug 08 15:55:27 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:27.147324 22128 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:55:27.148241 22128 webserver.cc:460] Webserver started at http://127.30.76.126:32973/ using document root <none> and password file <none>
I20260808 15:55:27.148732 22128 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:27.148792 22128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:27.152426 22128 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 15:55:27.154516 22149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.156095 22128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:55:27.156231 22128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "4468cec04050480785a89bc33b52dc77"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "b078cdcc3e0281f2b7fee27cdafc01be"
server_key_iv: "c19fce698957bb79ea29b4bbe4a47032"
server_key_version: "encryptionkey@0"
I20260808 15:55:27.156759 22128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:27.175294 22128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:27.178078 22128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:27.178292 22128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:27.185345 22128 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41345
I20260808 15:55:27.185653 22226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41345 every 8 connection(s)
I20260808 15:55:27.186398 22128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:27.190568 22227 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:55:27.190646 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 22128
I20260808 15:55:27.191080 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:44351
--local_ip_for_outbound_sockets=127.30.76.65
--tserver_master_addrs=127.30.76.126:41345
--webserver_port=39241
--webserver_interface=127.30.76.65
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:55:27.192366 22227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap starting.
I20260808 15:55:27.210096 22227 log.cc:824] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:27.221349 22227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:55:27.221765 22227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Bootstrap complete.
I20260808 15:55:27.225962 22227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:27.226378 22227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4468cec04050480785a89bc33b52dc77, State: Initialized, Role: FOLLOWER
I20260808 15:55:27.226876 22227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:27.227023 22227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:27.227113 22227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:27.227241 22227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 2 FOLLOWER]: Advancing to term 3
I20260808 15:55:27.229115 22227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:27.229445 22227 leader_election.cc:302] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4468cec04050480785a89bc33b52dc77; no voters:
I20260808 15:55:27.229782 22227 leader_election.cc:288] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 15:55:27.229882 22231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 15:55:27.230098 22231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [term 3 LEADER]: Becoming Leader. State: Replica: 4468cec04050480785a89bc33b52dc77, State: Running, Role: LEADER
I20260808 15:55:27.230381 22231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } }
I20260808 15:55:27.231010 22227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:27.231312 22231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4468cec04050480785a89bc33b52dc77. Latest consensus state: current_term: 3 leader_uuid: "4468cec04050480785a89bc33b52dc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } } }
I20260808 15:55:27.231439 22231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:27.231878 22232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4468cec04050480785a89bc33b52dc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4468cec04050480785a89bc33b52dc77" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41345 } } }
I20260808 15:55:27.232040 22232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:27.233105 22243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:27.238698 22243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:27.239351 22243 catalog_manager.cc:1259] Loaded cluster ID: facd9ae4c7014ac2b8aa479d7bbb8b92
I20260808 15:55:27.239444 22243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:27.241892 22243 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:27.242645 22243 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Loaded TSK: 0
I20260808 15:55:27.242758 22243 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4468cec04050480785a89bc33b52dc77: Loaded TSK: 1
F20260808 15:55:27.243489 22243 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0a41b3b9ad google::LogMessage::Fail() at ??:0
@ 0x7f0a41b3e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0a41b3b4de google::LogMessage::Flush() at ??:0
@ 0x7f0a41b3ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0a447ff18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0a447f7f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0a4c4a47bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0a4c47e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0a4c47f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f0a4c4c0df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f0a4c4df82b std::function<>::operator()() at ??:0
@ 0x7f0a4c47eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f0a4c47ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0a4c47c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f0a4c4c0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0a43d8d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0a43d8dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f0a43d8f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0a43d7e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0a453c26db start_thread at ??:0
@ 0x7f0a4063961f clone at ??:0
W20260808 15:55:27.335647 22229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:27.335868 22229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:27.335911 22229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:27.340195 22229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:27.340240 22229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:27.340296 22229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:27.345924 22229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=39241
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:27.346868 22229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:27.347887 22229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:27.357692 22303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:27.357643 22300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:27.357818 22229 server_base.cc:1034] running on GCE node
W20260808 15:55:27.357894 22305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:27.358361 22229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:27.359048 22229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:27.360224 22229 hybrid_clock.cc:648] HybridClock initialized: now 1786204527360198 us; error 46 us; skew 500 ppm
Aug 08 15:55:27 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:27.362914 22229 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:55:27.363890 22229 webserver.cc:460] Webserver started at http://127.30.76.65:39241/ using document root <none> and password file <none>
I20260808 15:55:27.364435 22229 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:27.364497 22229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:27.368299 22229 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:55:27.370500 22318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.371816 22229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:27.371930 22229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "551621d079294eb6b86158b6a380a69e"
format_stamp: "Formatted at 2026-08-08 15:54:44 on dist-test-slave-sb2z"
server_key: "ead126e6aec42e2a14ca1fdf69cedc6d"
server_key_iv: "28b3b7096741a5427c0e66301eb15a2a"
server_key_version: "encryptionkey@0"
I20260808 15:55:27.372439 22229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:27.397154 22229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:27.400301 22229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:27.400502 22229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:27.401124 22229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:27.402269 22330 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 15:55:27.407673 22229 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 15:55:27.407755 22229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 15:55:27.407842 22229 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 15:55:27.409467 22229 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 15:55:27.409536 22229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 15:55:27.409823 22330 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Bootstrap starting.
I20260808 15:55:27.420387 22229 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:44351
I20260808 15:55:27.420626 22499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:44351 every 8 connection(s)
I20260808 15:55:27.421422 22229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:27.429111 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 22229
I20260808 15:55:27.429564 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:43145
--local_ip_for_outbound_sockets=127.30.76.66
--tserver_master_addrs=127.30.76.126:41345
--webserver_port=39453
--webserver_interface=127.30.76.66
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:55:27.432050 22330 log.cc:824] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:27.438932 22330 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:55:27.439320 22330 tablet_bootstrap.cc:492] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Bootstrap complete.
I20260808 15:55:27.440483 22330 ts_tablet_manager.cc:1397] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Time spent bootstrapping tablet: real 0.031s user 0.019s sys 0.011s
I20260808 15:55:27.442751 22330 raft_consensus.cc:348] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:27.443176 22330 raft_consensus.cc:729] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551621d079294eb6b86158b6a380a69e, State: Initialized, Role: FOLLOWER
I20260808 15:55:27.443704 22330 consensus_queue.cc:260] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:27.444635 22330 ts_tablet_manager.cc:1428] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260808 15:55:27.512549 22502 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:55:27.423426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:42105)
0808 15:55:27.423960 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.424023 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:27.424845 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:27.511791 (+ 86946us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":195,"threads_started":1}
W20260808 15:55:27.513192 22500 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:41345 (0 consecutive failures): Network error: Failed to ping master at 127.30.76.126:41345: Client connection negotiation failed: client connection to 127.30.76.126:41345: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 15:55:27.513950 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.513686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:43423)
0808 15:55:27.513819 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.513895 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:55:27.514667 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.514454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:53417)
0808 15:55:27.514557 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.514603 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:55:27.514883 22500 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:55:27.567368 22503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:27.567687 22503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:27.567770 22503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:27.571841 22503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:27.571893 22503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:27.571987 22503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:27.577108 22503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:43145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=39453
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:27.578178 22503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:27.579187 22503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:27.588652 22516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:27.588666 22512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:27.588766 22503 server_base.cc:1034] running on GCE node
W20260808 15:55:27.588675 22511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:27.589314 22503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:27.589984 22503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:27.591171 22503 hybrid_clock.cc:648] HybridClock initialized: now 1786204527591160 us; error 32 us; skew 500 ppm
Aug 08 15:55:27 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:27.593885 22503 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:55:27.594633 22503 webserver.cc:460] Webserver started at http://127.30.76.66:39453/ using document root <none> and password file <none>
I20260808 15:55:27.595130 22503 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:27.595191 22503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:27.598760 22503 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:27.600693 22525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.601863 22503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:55:27.601958 22503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "3e25b5b6190c476e89109f688c0845e2"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "4ed0371fb1169e9dd705a76cd283cbf7"
server_key_iv: "30bccc92a6c77add3ec50ac95ffe130f"
server_key_version: "encryptionkey@0"
I20260808 15:55:27.602438 22503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:27.612429 22503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:27.614948 22503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:27.615089 22503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:27.615643 22503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:27.616374 22503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:27.616421 22503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.616460 22503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:27.616482 22503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.625234 22503 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:43145
I20260808 15:55:27.625322 22689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:43145 every 8 connection(s)
I20260808 15:55:27.626211 22503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:27.628033 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 22503
I20260808 15:55:27.628474 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:42957
--local_ip_for_outbound_sockets=127.30.76.67
--tserver_master_addrs=127.30.76.126:41345
--webserver_port=40915
--webserver_interface=127.30.76.67
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:55:27.630503 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.628287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:59361)
0808 15:55:27.628802 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.628942 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":120,"threads_started":1}
W20260808 15:55:27.631027 22691 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:41345 (0 consecutive failures): Network error: Failed to ping master at 127.30.76.126:41345: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
I20260808 15:55:27.631704 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.631451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:51771)
0808 15:55:27.631604 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.631651 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 15:55:27.632397 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.632113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:40037)
0808 15:55:27.632255 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.632317 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 15:55:27.632584 22691 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:55:27.740377 22695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:27.740608 22695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:27.740648 22695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:27.744550 22695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:27.744598 22695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:27.744653 22695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:27.749637 22695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:42957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=40915
--tserver_master_addrs=127.30.76.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:27.750590 22695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:27.751643 22695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:55:27.762295 22695 server_base.cc:1034] running on GCE node
W20260808 15:55:27.762259 22701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:27.762259 22702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:27.762265 22704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:27.762892 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:27.763496 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:27.764708 22695 hybrid_clock.cc:648] HybridClock initialized: now 1786204527764686 us; error 33 us; skew 500 ppm
Aug 08 15:55:27 dist-test-slave-sb2z krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:27.767374 22695 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:55:27.768361 22695 webserver.cc:460] Webserver started at http://127.30.76.67:40915/ using document root <none> and password file <none>
I20260808 15:55:27.769001 22695 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:27.769093 22695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:27.772867 22695 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:55:27.774901 22714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.776228 22695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:27.776355 22695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "b03f907bcbae4655be7a3675299ee5da"
format_stamp: "Formatted at 2026-08-08 15:54:45 on dist-test-slave-sb2z"
server_key: "6b5db5728dccffc61a5cd52f400cf6c8"
server_key_iv: "147a71a2c475ca1de8da06ab5a0f6ae9"
server_key_version: "encryptionkey@0"
I20260808 15:55:27.776947 22695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:27.788509 22695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:27.791198 22695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:27.791390 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:27.792011 22695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:27.792825 22695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:27.792893 22695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.792963 22695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:27.793012 22695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:27.801904 22695 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:42957
I20260808 15:55:27.801973 22877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:42957 every 8 connection(s)
I20260808 15:55:27.802897 22695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:27.806281 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 22695
I20260808 15:55:27.807019 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.804960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:54355)
0808 15:55:27.805451 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.805555 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
W20260808 15:55:27.807646 22878 heartbeater.cc:644] Failed to heartbeat to 127.30.76.126:41345 (0 consecutive failures): Network error: Failed to ping master at 127.30.76.126:41345: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
I20260808 15:55:27.808241 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.808009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:51767)
0808 15:55:27.808122 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.808178 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 15:55:27.808835 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:27.808653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:40491)
0808 15:55:27.808750 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:27.808790 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:55:27.809011 22878 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:55:28.516036 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:28.515472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:55243)
0808 15:55:28.515864 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:28.515921 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":124,"threads_started":1}
I20260808 15:55:28.633440 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:28.633068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:55663)
0808 15:55:28.633292 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:28.633342 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
I20260808 15:55:28.809852 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:28.809482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:39579)
0808 15:55:28.809703 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:28.809753 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260808 15:55:29.068226 22887 raft_consensus.cc:482] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:29.068413 22887 raft_consensus.cc:504] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:29.069566 22887 leader_election.cc:288] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
Aug 08 15:55:29 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.67@KRBTEST.COM
Aug 08 15:55:29 dist-test-slave-sb2z krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.66@KRBTEST.COM
I20260808 15:55:29.086468 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.069780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:43145 (local address 127.30.76.65:46905)
0808 15:55:29.070126 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:29.070143 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:29.070250 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:29.072454 (+ 2204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:29.072459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:29.072853 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:29.073685 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:29.073706 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.075596 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.075600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:29.077079 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:29.077090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.078049 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.078053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:29.078329 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:29.079060 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:29.079083 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:29.081414 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:29.083758 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:29.083761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:29.083764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:29.084039 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:29.084384 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:29.084387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:29.084389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:29.084602 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:29.085341 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:29.085347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:29.085698 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 15:55:29.085916 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 15:55:29.086258 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":409,"spinlock_wait_cycles":21248,"thread_start_us":72,"threads_started":1}
I20260808 15:55:29.086536 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.070005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:43099 (local address 127.30.76.67:42957)
0808 15:55:29.070459 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:29.070479 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:29.070544 (+ 65us) server_negotiation.cc:407] Connection header received
0808 15:55:29.071255 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:29.071260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:29.071649 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:29.071969 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:29.073880 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.075415 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.077212 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.077919 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.078089 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:29.081258 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:29.081263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:29.081274 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:29.081300 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:29.083365 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:29.084101 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:29.084110 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:29.084117 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:29.084279 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:29.084728 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:29.084732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:29.084733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:29.085217 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:29.085352 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:29.086064 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 15:55:29.086348 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":126,"threads_started":1}
I20260808 15:55:29.086536 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.069956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:46905 (local address 127.30.76.66:43145)
0808 15:55:29.070462 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:29.070540 (+ 78us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:29.070584 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:55:29.071252 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:29.071259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:29.071607 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:29.071993 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:29.073880 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.075415 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.077212 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.077919 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.078089 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:29.081644 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:29.081657 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:29.081668 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:29.081695 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:29.083656 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:29.084150 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:29.084156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:29.084161 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:29.084279 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:29.084719 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:29.084723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:29.084724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:29.085217 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:29.085353 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:29.086064 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 15:55:29.086348 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":130,"threads_started":1}
I20260808 15:55:29.086468 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.069822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:42957 (local address 127.30.76.65:43099)
0808 15:55:29.070226 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:29.070241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:29.070361 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:29.072140 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:29.072149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:29.072853 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:29.073685 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:29.073706 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.075596 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.075600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:29.077079 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:29.077090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:29.078049 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:29.078053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:29.078327 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:29.079069 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:29.079084 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:29.080996 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:29.083488 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:29.083493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:29.083504 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:29.083966 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:29.084384 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:29.084387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:29.084389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:29.084602 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:29.085341 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:29.085347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:29.085698 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 15:55:29.085916 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 15:55:29.086311 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":589,"spinlock_wait_cycles":24320,"thread_start_us":66,"threads_started":1}
W20260808 15:55:29.089757 22325 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
W20260808 15:55:29.089901 22319 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:29.089979 22319 leader_election.cc:302] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 551621d079294eb6b86158b6a380a69e; no voters: 3e25b5b6190c476e89109f688c0845e2, b03f907bcbae4655be7a3675299ee5da
I20260808 15:55:29.090204 22887 raft_consensus.cc:2726] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:55:29.517282 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.516900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:43167)
0808 15:55:29.517135 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:29.517194 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 15:55:29.634685 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.634307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:39701)
0808 15:55:29.634538 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:29.634590 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260808 15:55:29.810936 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:29.810536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:53307)
0808 15:55:29.810800 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:29.810851 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":109,"threads_started":1}
I20260808 15:55:30.518438 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:30.518028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:35797)
0808 15:55:30.518287 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:30.518340 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260808 15:55:30.635934 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:30.635455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:48647)
0808 15:55:30.635797 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:30.635850 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":92,"threads_started":1}
I20260808 15:55:30.812117 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:30.811691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:57525)
0808 15:55:30.811971 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:30.812027 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":77,"threads_started":1}
I20260808 15:55:31.025496 22906 raft_consensus.cc:482] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:31.025635 22906 raft_consensus.cc:504] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:31.026132 22906 leader_election.cc:288] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
W20260808 15:55:31.026955 22325 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
W20260808 15:55:31.027087 22319 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:31.027179 22319 leader_election.cc:302] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 551621d079294eb6b86158b6a380a69e; no voters: 3e25b5b6190c476e89109f688c0845e2, b03f907bcbae4655be7a3675299ee5da
I20260808 15:55:31.027346 22906 raft_consensus.cc:2726] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:55:31.519845 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:31.519242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:33371)
0808 15:55:31.519589 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:31.519664 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260808 15:55:31.637161 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:31.636712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:56633)
0808 15:55:31.636992 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:31.637077 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260808 15:55:31.813215 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:31.812820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:47461)
0808 15:55:31.813062 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:31.813114 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260808 15:55:32.521215 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:32.520733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:44205)
0808 15:55:32.521059 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:32.521116 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":75,"threads_started":1}
I20260808 15:55:32.638209 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:32.637833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:52473)
0808 15:55:32.638061 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:32.638113 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
I20260808 15:55:32.814325 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:32.813935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:45911)
0808 15:55:32.814180 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:32.814233 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260808 15:55:33.522449 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:33.522005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:44933)
0808 15:55:33.522298 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:33.522354 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":100,"threads_started":1}
I20260808 15:55:33.639358 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:33.638930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:38317)
0808 15:55:33.639215 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:33.639266 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":121,"threads_started":1}
I20260808 15:55:33.815616 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:33.815037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:46397)
0808 15:55:33.815391 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:33.815447 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":102,"threads_started":1}
I20260808 15:55:34.197721 22921 raft_consensus.cc:482] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:34.197849 22921 raft_consensus.cc:504] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:34.198339 22921 leader_election.cc:288] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
W20260808 15:55:34.199107 22319 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
W20260808 15:55:34.199229 22325 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:34.199322 22325 leader_election.cc:302] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 551621d079294eb6b86158b6a380a69e; no voters: 3e25b5b6190c476e89109f688c0845e2, b03f907bcbae4655be7a3675299ee5da
I20260808 15:55:34.199515 22921 raft_consensus.cc:2726] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:55:34.523605 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:34.523161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:43301)
0808 15:55:34.523400 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:34.523450 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260808 15:55:34.640583 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:34.640111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:38725)
0808 15:55:34.640431 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:34.640485 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":86,"threads_started":1}
I20260808 15:55:34.816710 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:34.816286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:56745)
0808 15:55:34.816564 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:34.816615 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":68,"threads_started":1}
I20260808 15:55:35.524731 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:35.524320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:52093)
0808 15:55:35.524583 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:35.524634 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260808 15:55:35.641902 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:35.641358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:57227)
0808 15:55:35.641700 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:35.641770 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260808 15:55:35.817893 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:35.817489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:51775)
0808 15:55:35.817749 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:35.817801 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":92,"threads_started":1}
I20260808 15:55:36.525889 22931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:36.525469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:35179)
0808 15:55:36.525739 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:36.525792 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260808 15:55:36.643117 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:36.642712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:48621)
0808 15:55:36.642971 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:36.643027 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260808 15:55:36.818985 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:36.818593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:35135)
0808 15:55:36.818838 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:36.818894 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
I20260808 15:55:37.527339 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:37.526910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:40351)
0808 15:55:37.527184 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:37.527241 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260808 15:55:37.644287 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:37.643894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:49195)
0808 15:55:37.644146 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:37.644199 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":87,"threads_started":1}
I20260808 15:55:37.820266 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:37.819831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:35459)
0808 15:55:37.820118 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:37.820173 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":89,"threads_started":1}
I20260808 15:55:38.528548 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:38.528094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:53089)
0808 15:55:38.528394 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:38.528452 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
I20260808 15:55:38.645334 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:38.644930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:38361)
0808 15:55:38.645184 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:38.645237 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":72,"threads_started":1}
I20260808 15:55:38.665827 22942 raft_consensus.cc:482] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:38.665954 22942 raft_consensus.cc:504] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e25b5b6190c476e89109f688c0845e2" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 43145 } } peers { permanent_uuid: "b03f907bcbae4655be7a3675299ee5da" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42957 } } peers { permanent_uuid: "551621d079294eb6b86158b6a380a69e" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44351 } }
I20260808 15:55:38.666419 22942 leader_election.cc:288] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145), b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957)
W20260808 15:55:38.667140 22319 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03f907bcbae4655be7a3675299ee5da (127.30.76.67:42957): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
W20260808 15:55:38.667223 22325 leader_election.cc:341] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e25b5b6190c476e89109f688c0845e2 (127.30.76.66:43145): Not found: Tablet not found: 5ba2edcde4fd47999ad62e4387b7ae5a
I20260808 15:55:38.667338 22325 leader_election.cc:302] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 551621d079294eb6b86158b6a380a69e; no voters: 3e25b5b6190c476e89109f688c0845e2, b03f907bcbae4655be7a3675299ee5da
I20260808 15:55:38.667577 22942 raft_consensus.cc:2726] T 5ba2edcde4fd47999ad62e4387b7ae5a P 551621d079294eb6b86158b6a380a69e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:55:38.821472 22943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:38.821038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:44131)
0808 15:55:38.821321 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:38.821379 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":97,"threads_started":1}
I20260808 15:55:39.529742 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:39.529332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:41101)
0808 15:55:39.529581 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:39.529644 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260808 15:55:39.646682 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:39.646163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:47297)
0808 15:55:39.646511 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:39.646567 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":89,"threads_started":1}
I20260808 15:55:39.822715 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:39.822252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:55815)
0808 15:55:39.822513 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:39.822568 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260808 15:55:40.530925 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:40.530502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:55851)
0808 15:55:40.530771 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:40.530829 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260808 15:55:40.647955 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:40.647515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:57867)
0808 15:55:40.647808 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:40.647861 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":71,"threads_started":1}
I20260808 15:55:40.823961 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:40.823511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:58921)
0808 15:55:40.823813 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:40.823868 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":74,"threads_started":1}
I20260808 15:55:41.532318 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:41.531844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:42967)
0808 15:55:41.532163 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:41.532218 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":136,"threads_started":1}
I20260808 15:55:41.649067 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:41.648684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:46967)
0808 15:55:41.648918 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:41.648974 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260808 15:55:41.825088 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:41.824665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:51253)
0808 15:55:41.824931 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:41.824992 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260808 15:55:42.533743 22961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:42.533328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.65:32975)
0808 15:55:42.533588 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:42.533645 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
I20260808 15:55:42.650384 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:42.649975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.66:43135)
0808 15:55:42.650237 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:42.650292 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260808 15:55:42.814806 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 22229
I20260808 15:55:42.826423 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:42.826018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41345 (local address 127.30.76.67:46831)
0808 15:55:42.826278 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:42.826333 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.126:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
I20260808 15:55:42.828037 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 22503
I20260808 15:55:42.838794 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 22695
2026-08-08T15:55:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59646 ms)
[----------] 30 tests from SecurityITest (247858 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:42 dist-test-slave-sb2z krb5kdc[22967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:42 dist-test-slave-sb2z krb5kdc[22967](info): set up 2 sockets
Aug 08 15:55:42 dist-test-slave-sb2z krb5kdc[22967](info): commencing operation
krb5kdc: starting...
W20260808 15:55:44.069082 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:55:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:55:44.194077 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36325
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:36325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:44.312235 22985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:44.312476 22985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:44.312515 22985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:44.316390 22985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:44.316442 22985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:44.316460 22985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:44.316476 22985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:44.316491 22985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:44.322299 22985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:36325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:36325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:44.323367 22985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:44.324496 22985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:44.333667 22994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:44.333678 22992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:44.333860 22985 server_base.cc:1034] running on GCE node
W20260808 15:55:44.334021 22991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:44.334709 22985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:44.336035 22985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:44.337275 22985 hybrid_clock.cc:648] HybridClock initialized: now 1786204544337247 us; error 47 us; skew 500 ppm
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:44.340446 22985 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:55:44.341462 22985 webserver.cc:460] Webserver started at http://127.30.76.126:40921/ using document root <none> and password file <none>
I20260808 15:55:44.342067 22985 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:44.342170 22985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:44.342414 22985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:44.344668 22985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "0f276c5bb5694432b1fa07170bbf62ee"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "26b052051f5db89c289340a0a9d13e44"
server_key_iv: "737f10a05cb7d06ce0e6122782823f30"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.345286 22985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "0f276c5bb5694432b1fa07170bbf62ee"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "26b052051f5db89c289340a0a9d13e44"
server_key_iv: "737f10a05cb7d06ce0e6122782823f30"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.349237 22985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:55:44.351876 23004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.353140 22985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:55:44.353302 22985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "0f276c5bb5694432b1fa07170bbf62ee"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "26b052051f5db89c289340a0a9d13e44"
server_key_iv: "737f10a05cb7d06ce0e6122782823f30"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.353416 22985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:44.367851 22985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:44.370756 22985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:44.370976 22985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:44.378530 22985 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:36325
I20260808 15:55:44.378549 23082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:36325 every 8 connection(s)
I20260808 15:55:44.379515 22985 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:44.380401 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 22985
I20260808 15:55:44.380535 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:44.380860 31025 external_mini_cluster.cc:1428] Setting key 0c9a782f357792b602b96a8a83fb146e
I20260808 15:55:44.382880 23083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:44.389539 23083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: Bootstrap starting.
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:44.392120 23083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:44.392911 23083 log.cc:824] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:44.395080 23083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: No bootstrap required, opened a new log
I20260808 15:55:44.395689 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:44.382267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.30.76.126:36325)
0808 15:55:44.382681 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:44.382701 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:44.382736 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:55:44.383393 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:44.383410 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:44.383795 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:44.384059 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:44.384992 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.385929 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.387094 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.387595 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.387724 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:44.390620 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:44.390625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:44.390637 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:44.390667 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:44.392653 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:44.393127 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:44.393138 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:44.393143 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:44.393255 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:44.393572 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:44.393575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:44.393577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:44.393963 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:44.394155 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:44.394525 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 15:55:44.394814 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":136,"threads_started":1}
I20260808 15:55:44.399387 23083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f276c5bb5694432b1fa07170bbf62ee" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36325 } }
I20260808 15:55:44.399711 23083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:44.399796 23083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f276c5bb5694432b1fa07170bbf62ee, State: Initialized, Role: FOLLOWER
I20260808 15:55:44.400269 23083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f276c5bb5694432b1fa07170bbf62ee" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36325 } }
I20260808 15:55:44.400440 23083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:44.400535 23083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:44.400641 23083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:44.401708 23083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f276c5bb5694432b1fa07170bbf62ee" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36325 } }
I20260808 15:55:44.402072 23083 leader_election.cc:302] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f276c5bb5694432b1fa07170bbf62ee; no voters:
I20260808 15:55:44.402464 23083 leader_election.cc:288] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:44.402630 23090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:44.402899 23090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [term 1 LEADER]: Becoming Leader. State: Replica: 0f276c5bb5694432b1fa07170bbf62ee, State: Running, Role: LEADER
I20260808 15:55:44.403314 23090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f276c5bb5694432b1fa07170bbf62ee" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36325 } }
I20260808 15:55:44.403857 23083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:44.405678 23091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f276c5bb5694432b1fa07170bbf62ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f276c5bb5694432b1fa07170bbf62ee" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36325 } } }
I20260808 15:55:44.405838 23091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:44.406536 23092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f276c5bb5694432b1fa07170bbf62ee. Latest consensus state: current_term: 1 leader_uuid: "0f276c5bb5694432b1fa07170bbf62ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f276c5bb5694432b1fa07170bbf62ee" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 36325 } } }
I20260808 15:55:44.406651 23092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee [sys.catalog]: This master's current role is: LEADER
W20260808 15:55:44.409392 23113 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:55:44.409511 23113 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:55:44.409597 23106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:44.410442 23106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:44.416172 23106 catalog_manager.cc:1347] Generated new cluster ID: 2c359e79de714ff6854385cba02c815c
I20260808 15:55:44.416244 23106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:44.436349 23106 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:44.437594 23106 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:44.442553 23106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: Generated new TSK 0
I20260808 15:55:44.443285 23106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:55:44.478806 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36325
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:44.597677 23119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:44.597985 23119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:44.598078 23119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:44.602619 23119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:44.602712 23119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:44.602804 23119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:44.609380 23119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:44.610515 23119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:44.611634 23119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:44.621791 23124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:44.621868 23119 server_base.cc:1034] running on GCE node
W20260808 15:55:44.621798 23126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:44.621798 23128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:44.622454 23119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:44.623054 23119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:44.624245 23119 hybrid_clock.cc:648] HybridClock initialized: now 1786204544624237 us; error 28 us; skew 500 ppm
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:44.627039 23119 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:55:44.627988 23119 webserver.cc:460] Webserver started at http://127.30.76.65:38509/ using document root <none> and password file <none>
I20260808 15:55:44.628474 23119 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:44.628530 23119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:44.628696 23119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:44.630549 23119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "6897923c23b4446ab3c5ac076bfc1949"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "3bf4a18d0736a3e63d86b478bb8c25cb"
server_key_iv: "9910da69e9bed2666450a8e3f7ae00ae"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.631083 23119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "6897923c23b4446ab3c5ac076bfc1949"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "3bf4a18d0736a3e63d86b478bb8c25cb"
server_key_iv: "9910da69e9bed2666450a8e3f7ae00ae"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.634560 23119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:55:44.636833 23138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.637965 23119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:55:44.638088 23119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "6897923c23b4446ab3c5ac076bfc1949"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "3bf4a18d0736a3e63d86b478bb8c25cb"
server_key_iv: "9910da69e9bed2666450a8e3f7ae00ae"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.638193 23119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:44.651165 23119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:44.654320 23119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:44.654456 23119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:44.655017 23119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:44.655862 23119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:44.655910 23119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.655980 23119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:44.656024 23119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.665146 23119 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:45359
I20260808 15:55:44.665198 23307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:45359 every 8 connection(s)
I20260808 15:55:44.665998 23119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:44.675896 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 23119
I20260808 15:55:44.676069 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:55:44.676395 31025 external_mini_cluster.cc:1428] Setting key 11de8ba72d1c89cc17ac9e5291a60fe1
I20260808 15:55:44.679265 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:44.667749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:35235 (local address 127.30.76.126:36325)
0808 15:55:44.667890 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:44.667894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:44.668906 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 15:55:44.669765 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:44.669769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:44.669829 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:44.669945 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:44.671062 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.671700 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.672852 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.673117 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.673242 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:44.675702 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:44.675705 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:44.675707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:44.675733 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:44.677186 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:44.677714 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:44.677718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:44.677720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:44.677804 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:44.678187 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:44.678191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:44.678193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:44.678378 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:44.678495 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:44.678973 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 15:55:44.679103 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:55:44.680168 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:44.668110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36325 (local address 127.30.76.65:35235)
0808 15:55:44.668738 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:44.668776 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:44.669556 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:44.670110 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:44.670117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:44.670467 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:44.670903 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:44.670915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.671834 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.671837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:44.672703 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:44.672713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.673284 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.673288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:44.673463 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:44.674026 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:44.674042 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:44.675484 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:44.677316 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:44.677322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:44.677336 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:44.677595 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:44.677916 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:44.677919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:44.677921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:44.678062 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:44.678495 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:44.678503 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:44.678741 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 15:55:44.678930 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 15:55:44.679251 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":102,"threads_started":1}
I20260808 15:55:44.681697 23308 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36325
I20260808 15:55:44.681943 23308 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:44.682443 23308 heartbeater.cc:511] Master 127.30.76.126:36325 requested a full tablet report, sending...
I20260808 15:55:44.684046 23026 ts_manager.cc:194] Registered new tserver with Master: 6897923c23b4446ab3c5ac076bfc1949 (127.30.76.65:45359)
I20260808 15:55:44.685194 23026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:35235
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:55:44.706578 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36325
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:44.824079 23317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:44.824311 23317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:44.824350 23317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:44.828262 23317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:44.828315 23317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:44.828368 23317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:44.833407 23317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:44.834336 23317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:44.835331 23317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:44.845067 23328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:44.845080 23325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:44.845158 23326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:44.845526 23317 server_base.cc:1034] running on GCE node
I20260808 15:55:44.845920 23317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:44.846570 23317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:44.847765 23317 hybrid_clock.cc:648] HybridClock initialized: now 1786204544847743 us; error 31 us; skew 500 ppm
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:44.851202 23317 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:55:44.852092 23317 webserver.cc:460] Webserver started at http://127.30.76.66:40457/ using document root <none> and password file <none>
I20260808 15:55:44.852622 23317 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:44.852679 23317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:44.852833 23317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:44.854727 23317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "9a72cb099cd145cb9755f11bcd34029e"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "6615fcb9ce3e760bce685848e442ec74"
server_key_iv: "c55d1c6b23c72a7e41d5bc0572793d1a"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.855214 23317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "9a72cb099cd145cb9755f11bcd34029e"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "6615fcb9ce3e760bce685848e442ec74"
server_key_iv: "c55d1c6b23c72a7e41d5bc0572793d1a"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.859265 23317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:55:44.861706 23342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.862866 23317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:44.862987 23317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "9a72cb099cd145cb9755f11bcd34029e"
format_stamp: "Formatted at 2026-08-08 15:55:44 on dist-test-slave-sb2z"
server_key: "6615fcb9ce3e760bce685848e442ec74"
server_key_iv: "c55d1c6b23c72a7e41d5bc0572793d1a"
server_key_version: "encryptionkey@0"
I20260808 15:55:44.863112 23317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:44.873073 23317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:44.876288 23317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:44.876497 23317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:44.877060 23317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:44.877853 23317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:44.877928 23317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.877985 23317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:44.878023 23317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:44.887185 23317 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:42547
I20260808 15:55:44.887212 23511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:42547 every 8 connection(s)
I20260808 15:55:44.888142 23317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:44.893735 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 23317
I20260808 15:55:44.893898 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:55:44.894280 31025 external_mini_cluster.cc:1428] Setting key 4c3fd693e4145c21e4427262ce68c65e
Aug 08 15:55:44 dist-test-slave-sb2z krb5kdc[22967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204544, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:44.902658 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:44.889917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:46701 (local address 127.30.76.126:36325)
0808 15:55:44.890097 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:44.890101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:44.891163 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 15:55:44.892107 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:44.892110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:44.892166 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:44.892240 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:44.893349 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.894008 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.895594 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.895833 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.895960 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:44.898654 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:44.898657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:44.898660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:44.898682 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:44.900269 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:44.900944 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:44.900946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:44.900948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:44.901025 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:44.901444 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:44.901446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:44.901447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:44.901618 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:44.901706 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:44.902422 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 15:55:44.902526 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 15:55:44.903995 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:44.890289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36325 (local address 127.30.76.66:46701)
0808 15:55:44.891009 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:44.891049 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:44.891918 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:44.892403 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:44.892409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:44.892760 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:44.893221 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:44.893234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.894168 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.894172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:44.895432 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:44.895443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:44.895999 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:44.896003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:44.896184 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:44.896825 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:44.896841 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:44.898462 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:44.900403 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:44.900414 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:44.900434 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:44.900823 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:44.901133 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:44.901145 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:44.901147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:44.901342 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:44.901754 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:44.901763 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:44.902122 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 15:55:44.902387 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 15:55:44.902780 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":117,"threads_started":1}
I20260808 15:55:44.905333 23512 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36325
I20260808 15:55:44.905642 23512 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:55:44.906219 23512 heartbeater.cc:511] Master 127.30.76.126:36325 requested a full tablet report, sending...
I20260808 15:55:44.907496 23026 ts_manager.cc:194] Registered new tserver with Master: 9a72cb099cd145cb9755f11bcd34029e (127.30.76.66:42547)
I20260808 15:55:44.908208 23026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:46701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:55:44.926231 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36325
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:45.039214 23518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:45.039475 23518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:45.039518 23518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:45.043875 23518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:45.043929 23518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:45.043993 23518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:45.049166 23518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:36325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:45.050119 23518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:45.051067 23518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:45.060578 23525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:45.060557 23526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:45.060602 23528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:45.060803 23518 server_base.cc:1034] running on GCE node
I20260808 15:55:45.061304 23518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:45.061923 23518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:45.063108 23518 hybrid_clock.cc:648] HybridClock initialized: now 1786204545063055 us; error 43 us; skew 500 ppm
Aug 08 15:55:45 dist-test-slave-sb2z krb5kdc[22967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204545, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:45.066104 23518 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:55:45.067078 23518 webserver.cc:460] Webserver started at http://127.30.76.67:34345/ using document root <none> and password file <none>
I20260808 15:55:45.067726 23518 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:45.067791 23518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:45.067956 23518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:45.069939 23518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "ac21c99f70d249edbce359bead26db15"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "0bac2c750a2733b0ff6a527b1cda55f5"
server_key_iv: "e3210eeddda176c136b0692c772e665b"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.070513 23518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "ac21c99f70d249edbce359bead26db15"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "0bac2c750a2733b0ff6a527b1cda55f5"
server_key_iv: "e3210eeddda176c136b0692c772e665b"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.074095 23518 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 15:55:45.076467 23537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.077673 23518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:45.077797 23518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "ac21c99f70d249edbce359bead26db15"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "0bac2c750a2733b0ff6a527b1cda55f5"
server_key_iv: "e3210eeddda176c136b0692c772e665b"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.077901 23518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:45.092514 23518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:45.095237 23518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:45.095432 23518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:45.096014 23518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:45.096796 23518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:45.096850 23518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.096943 23518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:45.096982 23518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.105991 23518 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:42675
I20260808 15:55:45.106063 23706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:42675 every 8 connection(s)
I20260808 15:55:45.106849 23518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:45.113400 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 23518
I20260808 15:55:45.113561 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:55:45.113885 31025 external_mini_cluster.cc:1428] Setting key 2186065f200d199ad540785136f07fdf
Aug 08 15:55:45 dist-test-slave-sb2z krb5kdc[22967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204545, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:45.120283 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.108573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:51487 (local address 127.30.76.126:36325)
0808 15:55:45.108720 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.108724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.109748 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 15:55:45.110540 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.110542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.110589 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.110664 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:45.111829 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.112346 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.113661 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.113886 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.114021 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.116607 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:45.116615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:45.116617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:45.116641 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:45.118198 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:45.118723 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:45.118728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:45.118730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:45.118814 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:45.119170 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:45.119174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:45.119175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:45.119353 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:45.119472 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.119989 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.120105 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:55:45.121035 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.108940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:36325 (local address 127.30.76.67:51487)
0808 15:55:45.109583 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.109640 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.110362 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.110822 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.110828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.111159 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:45.111700 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.111715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.112452 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.112454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.113463 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.113476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.114077 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.114081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.114270 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.114923 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:45.114939 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:45.116438 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:45.118329 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:45.118335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:45.118348 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:45.118604 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:45.118916 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:45.118919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:45.118921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:45.119060 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:45.119468 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:45.119475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:45.119756 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.119946 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.120245 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":112,"threads_started":1}
I20260808 15:55:45.122120 23708 heartbeater.cc:347] Connected to a master server at 127.30.76.126:36325
I20260808 15:55:45.122352 23708 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:45.122822 23708 heartbeater.cc:511] Master 127.30.76.126:36325 requested a full tablet report, sending...
I20260808 15:55:45.123858 23026 ts_manager.cc:194] Registered new tserver with Master: ac21c99f70d249edbce359bead26db15 (127.30.76.67:42675)
I20260808 15:55:45.124379 23026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:51487
I20260808 15:55:45.127785 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:45.136430 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.129669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.30.76.126:36325)
0808 15:55:45.129796 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.129799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.129992 (+ 193us) server_negotiation.cc:407] Connection header received
0808 15:55:45.130095 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.130097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.130132 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.130198 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:45.130830 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.131305 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.132170 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.132370 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.132447 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.133678 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:45.133680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:45.133682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:45.133703 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:45.134890 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:45.135277 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:45.135279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:45.135280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:45.135349 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:45.135653 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:45.135655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:45.135656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:45.135849 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:45.135950 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.136162 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.136250 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:55:45.139091 23026 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:45.142258 23026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:45.156740 23728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.150606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:45359 (local address 127.0.0.1:35386)
0808 15:55:45.151552 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.151566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.152151 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.152806 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.152809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.152830 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.152920 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.152926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.154297 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.154300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.155225 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.155236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.156169 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.156173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.156287 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.156327 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.156473 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.156593 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":37,"thread_start_us":115,"threads_started":1}
I20260808 15:55:45.156750 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.150775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.30.76.65:45359)
0808 15:55:45.151385 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.151401 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.151669 (+ 268us) server_negotiation.cc:407] Connection header received
0808 15:55:45.152318 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.152323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.152473 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.152589 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.153066 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.154150 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.155368 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.156022 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.156153 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.156309 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.156505 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.156605 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":88,"threads_started":1}
I20260808 15:55:45.157397 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.151807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.30.76.66:42547)
0808 15:55:45.152121 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.152132 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.152182 (+ 50us) server_negotiation.cc:407] Connection header received
0808 15:55:45.152264 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.152270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.152448 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.152590 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.152791 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.153606 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.155372 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.156220 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.156328 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.156462 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.157168 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.157267 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":129,"threads_started":1}
I20260808 15:55:45.157454 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.150617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.30.76.67:42675)
0808 15:55:45.151187 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.151202 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.151583 (+ 381us) server_negotiation.cc:407] Connection header received
0808 15:55:45.151827 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.151833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.152058 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.152249 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.152538 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.153721 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.155372 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.156220 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.156328 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.156462 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.157234 (+ 772us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.157320 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":137,"threads_started":1}
I20260808 15:55:45.157099 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.151294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:42547 (local address 127.0.0.1:39812)
0808 15:55:45.151838 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.151851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.151928 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.152575 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.152580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.152603 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.152693 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.152699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.153745 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.153751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.155226 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.155235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.156530 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.156534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.156710 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.156764 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.156928 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.157000 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":182,"thread_start_us":93,"threads_started":1}
I20260808 15:55:45.157085 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.150507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:42675 (local address 127.0.0.1:53174)
0808 15:55:45.151324 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.151378 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.151515 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.152239 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.152244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.152266 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.152417 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.152424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.153965 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.153968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.155223 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.155235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.156531 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.156534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.156706 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.156763 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.156837 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.156925 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":59,"thread_start_us":98,"threads_started":1}
I20260808 15:55:45.160007 23207 tablet_service.cc:1467] Processing CreateTablet for tablet cad2cc52805247c8bc5f20323d9bcedc (DEFAULT_TABLE table=test-table [id=4369ae917b8c4558b8605ad85c526493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:45.160291 23409 tablet_service.cc:1467] Processing CreateTablet for tablet cad2cc52805247c8bc5f20323d9bcedc (DEFAULT_TABLE table=test-table [id=4369ae917b8c4558b8605ad85c526493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:45.161159 23207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cad2cc52805247c8bc5f20323d9bcedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:45.161387 23409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cad2cc52805247c8bc5f20323d9bcedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:45.162985 23609 tablet_service.cc:1467] Processing CreateTablet for tablet cad2cc52805247c8bc5f20323d9bcedc (DEFAULT_TABLE table=test-table [id=4369ae917b8c4558b8605ad85c526493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:45.163964 23609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cad2cc52805247c8bc5f20323d9bcedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:45.167445 23737 tablet_bootstrap.cc:492] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Bootstrap starting.
I20260808 15:55:45.169064 23738 tablet_bootstrap.cc:492] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949: Bootstrap starting.
I20260808 15:55:45.170455 23737 tablet_bootstrap.cc:654] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:45.170794 23739 tablet_bootstrap.cc:492] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Bootstrap starting.
I20260808 15:55:45.171211 23738 tablet_bootstrap.cc:654] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:45.171348 23737 log.cc:824] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:45.172480 23738 log.cc:824] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:45.172871 23739 tablet_bootstrap.cc:654] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:45.173560 23737 tablet_bootstrap.cc:492] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: No bootstrap required, opened a new log
I20260808 15:55:45.173748 23737 ts_tablet_manager.cc:1397] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:55:45.174047 23739 log.cc:824] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:45.174585 23738 tablet_bootstrap.cc:492] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949: No bootstrap required, opened a new log
I20260808 15:55:45.174824 23738 ts_tablet_manager.cc:1397] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 15:55:45.175948 23739 tablet_bootstrap.cc:492] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: No bootstrap required, opened a new log
I20260808 15:55:45.176133 23739 ts_tablet_manager.cc:1397] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 15:55:45.177497 23737 raft_consensus.cc:348] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.177773 23737 raft_consensus.cc:374] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:45.177872 23737 raft_consensus.cc:729] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a72cb099cd145cb9755f11bcd34029e, State: Initialized, Role: FOLLOWER
I20260808 15:55:45.177675 23738 raft_consensus.cc:348] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.177953 23738 raft_consensus.cc:374] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:45.178015 23738 raft_consensus.cc:729] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6897923c23b4446ab3c5ac076bfc1949, State: Initialized, Role: FOLLOWER
I20260808 15:55:45.178356 23737 consensus_queue.cc:260] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.178438 23738 consensus_queue.cc:260] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.179010 23512 heartbeater.cc:503] Master 127.30.76.126:36325 was elected leader, sending a full tablet report...
I20260808 15:55:45.179322 23737 ts_tablet_manager.cc:1428] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:55:45.179764 23739 raft_consensus.cc:348] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.179924 23308 heartbeater.cc:503] Master 127.30.76.126:36325 was elected leader, sending a full tablet report...
I20260808 15:55:45.180015 23739 raft_consensus.cc:374] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:45.180075 23739 raft_consensus.cc:729] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac21c99f70d249edbce359bead26db15, State: Initialized, Role: FOLLOWER
I20260808 15:55:45.180510 23739 consensus_queue.cc:260] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.180724 23738 ts_tablet_manager.cc:1428] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260808 15:55:45.181878 23708 heartbeater.cc:503] Master 127.30.76.126:36325 was elected leader, sending a full tablet report...
I20260808 15:55:45.182209 23739 ts_tablet_manager.cc:1428] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:55:45.280037 23746 raft_consensus.cc:482] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:45.280292 23746 raft_consensus.cc:504] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.281533 23746 leader_election.cc:288] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac21c99f70d249edbce359bead26db15 (127.30.76.67:42675), 9a72cb099cd145cb9755f11bcd34029e (127.30.76.66:42547)
I20260808 15:55:45.284315 23747 raft_consensus.cc:482] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:45.284504 23747 raft_consensus.cc:504] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.285492 23747 leader_election.cc:288] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6897923c23b4446ab3c5ac076bfc1949 (127.30.76.65:45359), 9a72cb099cd145cb9755f11bcd34029e (127.30.76.66:42547)
I20260808 15:55:45.287091 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.281924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:36949 (local address 127.30.76.66:42547)
0808 15:55:45.282061 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.282066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.282329 (+ 263us) server_negotiation.cc:407] Connection header received
0808 15:55:45.282389 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.282392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.282448 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.282534 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.282920 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.283737 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.285679 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.286289 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.286382 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.286433 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.286881 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.286951 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 15:55:45.287252 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.281822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:48571 (local address 127.30.76.67:42675)
0808 15:55:45.282076 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.282083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.282238 (+ 155us) server_negotiation.cc:407] Connection header received
0808 15:55:45.282390 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.282393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.282445 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.282532 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.282920 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.283724 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.285672 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.286289 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.286382 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.286433 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.287057 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.287139 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260808 15:55:45.287566 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.281776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:42547 (local address 127.30.76.65:36949)
0808 15:55:45.282221 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.282233 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.282310 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.282573 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.282576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.282620 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.282792 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.282800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.283851 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.283855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.285524 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.285530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.286566 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.286572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.286720 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.286764 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.287279 (+ 515us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.287374 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":195,"spinlock_wait_cycles":9216,"thread_start_us":129,"threads_started":1}
I20260808 15:55:45.287739 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.281759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:42675 (local address 127.30.76.65:48571)
0808 15:55:45.282123 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.282151 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.282278 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.282573 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.282576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.282619 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.282805 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.282810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.283851 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.283855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.285518 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.285530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.286532 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.286536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.286650 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.286687 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.287415 (+ 728us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.287490 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":225,"spinlock_wait_cycles":27648,"thread_start_us":106,"threads_started":1}
I20260808 15:55:45.288321 23444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad2cc52805247c8bc5f20323d9bcedc" candidate_uuid: "6897923c23b4446ab3c5ac076bfc1949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a72cb099cd145cb9755f11bcd34029e" is_pre_election: true
I20260808 15:55:45.288290 23636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad2cc52805247c8bc5f20323d9bcedc" candidate_uuid: "6897923c23b4446ab3c5ac076bfc1949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac21c99f70d249edbce359bead26db15" is_pre_election: true
I20260808 15:55:45.288550 23636 raft_consensus.cc:2454] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6897923c23b4446ab3c5ac076bfc1949 in term 0.
I20260808 15:55:45.288614 23444 raft_consensus.cc:2454] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6897923c23b4446ab3c5ac076bfc1949 in term 0.
I20260808 15:55:45.288996 23139 leader_election.cc:302] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6897923c23b4446ab3c5ac076bfc1949, ac21c99f70d249edbce359bead26db15; no voters:
I20260808 15:55:45.289297 23746 raft_consensus.cc:2781] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:45.289383 23746 raft_consensus.cc:482] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:55:45.289425 23746 raft_consensus.cc:3037] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:45.290457 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.285635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:45359 (local address 127.30.76.67:53561)
0808 15:55:45.285876 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.285893 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.285966 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.286824 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.286830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.286851 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.286948 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.286954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.288006 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.288009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.289152 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.289163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.289991 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.289995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.290093 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.290164 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.290280 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.290349 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":3}
I20260808 15:55:45.290452 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.286125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:53561 (local address 127.30.76.65:45359)
0808 15:55:45.286532 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.286536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.286547 (+ 11us) server_negotiation.cc:407] Connection header received
0808 15:55:45.286579 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.286582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.286642 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.286717 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.287070 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.287777 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.289294 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.289862 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.289975 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.290015 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.290294 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.290357 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333}
I20260808 15:55:45.290791 23746 raft_consensus.cc:504] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.290999 23239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad2cc52805247c8bc5f20323d9bcedc" candidate_uuid: "ac21c99f70d249edbce359bead26db15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6897923c23b4446ab3c5ac076bfc1949" is_pre_election: true
I20260808 15:55:45.291175 23746 leader_election.cc:288] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [CANDIDATE]: Term 1 election: Requested vote from peers ac21c99f70d249edbce359bead26db15 (127.30.76.67:42675), 9a72cb099cd145cb9755f11bcd34029e (127.30.76.66:42547)
I20260808 15:55:45.291262 23239 raft_consensus.cc:2379] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac21c99f70d249edbce359bead26db15 in current term 1: Already voted for candidate 6897923c23b4446ab3c5ac076bfc1949 in this term.
I20260808 15:55:45.291626 23636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad2cc52805247c8bc5f20323d9bcedc" candidate_uuid: "6897923c23b4446ab3c5ac076bfc1949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac21c99f70d249edbce359bead26db15"
I20260808 15:55:45.291735 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.285777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:42547 (local address 127.30.76.67:52137)
0808 15:55:45.286415 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.286429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.286537 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.286886 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.286890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.286908 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:45.287019 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.287025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.289205 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.289208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.290205 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.290215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.291231 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.291234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.291331 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.291362 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.291432 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.291550 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":83,"thread_start_us":99,"threads_started":1}
I20260808 15:55:45.291759 23636 raft_consensus.cc:3037] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:45.291849 23444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad2cc52805247c8bc5f20323d9bcedc" candidate_uuid: "6897923c23b4446ab3c5ac076bfc1949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a72cb099cd145cb9755f11bcd34029e"
I20260808 15:55:45.291949 23444 raft_consensus.cc:3037] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:45.292100 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.286116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:52137 (local address 127.30.76.66:42547)
0808 15:55:45.286436 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.286447 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.286614 (+ 167us) server_negotiation.cc:407] Connection header received
0808 15:55:45.286661 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.286663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.286704 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.286778 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:45.287395 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.289082 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.290342 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.290985 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.291097 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.291146 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.291879 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.291953 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":105,"threads_started":1}
I20260808 15:55:45.292490 23443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad2cc52805247c8bc5f20323d9bcedc" candidate_uuid: "ac21c99f70d249edbce359bead26db15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a72cb099cd145cb9755f11bcd34029e" is_pre_election: true
I20260808 15:55:45.292932 23542 leader_election.cc:302] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac21c99f70d249edbce359bead26db15; no voters: 6897923c23b4446ab3c5ac076bfc1949, 9a72cb099cd145cb9755f11bcd34029e
I20260808 15:55:45.293043 23636 raft_consensus.cc:2454] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6897923c23b4446ab3c5ac076bfc1949 in term 1.
I20260808 15:55:45.293159 23747 raft_consensus.cc:2726] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 15:55:45.293207 23444 raft_consensus.cc:2454] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6897923c23b4446ab3c5ac076bfc1949 in term 1.
I20260808 15:55:45.293432 23139 leader_election.cc:302] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6897923c23b4446ab3c5ac076bfc1949, ac21c99f70d249edbce359bead26db15; no voters:
I20260808 15:55:45.293612 23746 raft_consensus.cc:2781] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:45.293795 23746 raft_consensus.cc:686] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [term 1 LEADER]: Becoming Leader. State: Replica: 6897923c23b4446ab3c5ac076bfc1949, State: Running, Role: LEADER
I20260808 15:55:45.294097 23746 consensus_queue.cc:237] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } }
I20260808 15:55:45.297338 23026 catalog_manager.cc:5495] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6897923c23b4446ab3c5ac076bfc1949 (127.30.76.65). New cstate: current_term: 1 leader_uuid: "6897923c23b4446ab3c5ac076bfc1949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6897923c23b4446ab3c5ac076bfc1949" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 45359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 } health_report { overall_health: UNKNOWN } } }
I20260808 15:55:45.310487 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.306895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.30.76.65:45359)
0808 15:55:45.307037 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.307042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.307057 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:55:45.307260 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.307263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.307315 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.307423 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:45.307720 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.308206 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.309071 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.309293 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.309407 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.309730 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:45.310102 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:45.310240 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.310303 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.310351 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:55:45.316148 23636 raft_consensus.cc:1261] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 1 FOLLOWER]: Refusing update from remote peer 6897923c23b4446ab3c5ac076bfc1949: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:45.316205 23444 raft_consensus.cc:1261] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 1 FOLLOWER]: Refusing update from remote peer 6897923c23b4446ab3c5ac076bfc1949: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:45.316879 23746 consensus_queue.cc:1035] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac21c99f70d249edbce359bead26db15" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:45.317183 23752 consensus_queue.cc:1035] T cad2cc52805247c8bc5f20323d9bcedc P 6897923c23b4446ab3c5ac076bfc1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a72cb099cd145cb9755f11bcd34029e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 42547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:45.325794 23757 mvcc.cc:205] Tried to move back new op lower bound from 7316293817607311360 to 7316293817527345152. Current Snapshot: MvccSnapshot[applied={T|T < 7316293817607311360}]
I20260808 15:55:45.329831 23759 mvcc.cc:205] Tried to move back new op lower bound from 7316293817607311360 to 7316293817527345152. Current Snapshot: MvccSnapshot[applied={T|T < 7316293817607311360}]
I20260808 15:55:45.338035 23023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:45.338214 23023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:45.340811 23023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f276c5bb5694432b1fa07170bbf62ee: Sending DeleteTablet for 3 replicas of tablet cad2cc52805247c8bc5f20323d9bcedc
I20260808 15:55:45.341691 23409 tablet_service.cc:1514] Processing DeleteTablet for tablet cad2cc52805247c8bc5f20323d9bcedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:39812
I20260808 15:55:45.341658 23207 tablet_service.cc:1514] Processing DeleteTablet for tablet cad2cc52805247c8bc5f20323d9bcedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:35386
I20260808 15:55:45.341979 23609 tablet_service.cc:1514] Processing DeleteTablet for tablet cad2cc52805247c8bc5f20323d9bcedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:45 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:53174
I20260808 15:55:45.342486 23779 tablet_replica.cc:330] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: stopping tablet replica
I20260808 15:55:45.342832 23779 raft_consensus.cc:2229] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:45.342988 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 23119
I20260808 15:55:45.348162 23777 tablet_replica.cc:330] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: stopping tablet replica
I20260808 15:55:45.348218 23779 raft_consensus.cc:2258] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:45.348469 23777 raft_consensus.cc:2229] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:45.348734 23777 raft_consensus.cc:2258] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:45.350025 23779 ts_tablet_manager.cc:1905] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:45.352044 23777 ts_tablet_manager.cc:1905] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:45.353310 23779 ts_tablet_manager.cc:1918] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:45.353447 23779 log.cc:1196] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/cad2cc52805247c8bc5f20323d9bcedc
I20260808 15:55:45.353772 23779 ts_tablet_manager.cc:1939] T cad2cc52805247c8bc5f20323d9bcedc P ac21c99f70d249edbce359bead26db15: Deleting consensus metadata
I20260808 15:55:45.354931 23005 catalog_manager.cc:4841] TS ac21c99f70d249edbce359bead26db15 (127.30.76.67:42675): tablet cad2cc52805247c8bc5f20323d9bcedc (table test-table [id=4369ae917b8c4558b8605ad85c526493]) successfully deleted
I20260808 15:55:45.355603 23777 ts_tablet_manager.cc:1918] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:45.355724 23777 log.cc:1196] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/cad2cc52805247c8bc5f20323d9bcedc
I20260808 15:55:45.356431 23777 ts_tablet_manager.cc:1939] T cad2cc52805247c8bc5f20323d9bcedc P 9a72cb099cd145cb9755f11bcd34029e: Deleting consensus metadata
I20260808 15:55:45.357810 23008 catalog_manager.cc:4841] TS 9a72cb099cd145cb9755f11bcd34029e (127.30.76.66:42547): tablet cad2cc52805247c8bc5f20323d9bcedc (table test-table [id=4369ae917b8c4558b8605ad85c526493]) successfully deleted
I20260808 15:55:45.360298 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 23317
W20260808 15:55:45.360296 23009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:45359 (error 108)
I20260808 15:55:45.362835 23728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.362465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:45359 (local address 127.0.0.1:35392)
0808 15:55:45.362666 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.362738 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260808 15:55:45.362988 23009 catalog_manager.cc:4584] TS 6897923c23b4446ab3c5ac076bfc1949 (127.30.76.65:45359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cad2cc52805247c8bc5f20323d9bcedc: Network error: Client connection negotiation failed: client connection to 127.30.76.65:45359: connect: Connection refused (error 111)
I20260808 15:55:45.369647 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 23518
I20260808 15:55:45.382874 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 22985
2026-08-08T15:55:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2543 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T15:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:45Z Disabled control of system clock
I20260808 15:55:45.423051 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:44499
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:44499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:55:45.533671 23787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:45.533995 23787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:45.534068 23787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:45.538086 23787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:45.538197 23787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:45.538229 23787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:45.538264 23787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:45.538293 23787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:45.543080 23787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:44499
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:44499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:45.544240 23787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:45.545645 23787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:45.554045 23795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:45.554124 23798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:45.554250 23787 server_base.cc:1034] running on GCE node
W20260808 15:55:45.554055 23794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:45.554924 23787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:45.556047 23787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:45.557250 23787 hybrid_clock.cc:648] HybridClock initialized: now 1786204545557217 us; error 44 us; skew 500 ppm
I20260808 15:55:45.562171 23787 webserver.cc:460] Webserver started at http://127.30.76.126:46699/ using document root <none> and password file <none>
I20260808 15:55:45.562762 23787 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:45.562862 23787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:45.563112 23787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:45.565234 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "2913fd13f4f14c57862851285f415a38"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "0c67c2b00c3407742183a3978c1f9ec9"
server_key_iv: "6dc942710f251e9847d638795fc5c548"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.565850 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "2913fd13f4f14c57862851285f415a38"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "0c67c2b00c3407742183a3978c1f9ec9"
server_key_iv: "6dc942710f251e9847d638795fc5c548"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.569634 23787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260808 15:55:45.572022 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.573259 23787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:45.573408 23787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "2913fd13f4f14c57862851285f415a38"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "0c67c2b00c3407742183a3978c1f9ec9"
server_key_iv: "6dc942710f251e9847d638795fc5c548"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.573524 23787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:45.585583 23787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:45.586238 23787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:45.586437 23787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:45.593504 23787 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:44499
I20260808 15:55:45.593516 23887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:44499 every 8 connection(s)
I20260808 15:55:45.594415 23787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:45.597446 23888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:45.599279 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 23787
I20260808 15:55:45.599399 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:45.599747 31025 external_mini_cluster.cc:1428] Setting key 264de89a261e2d5e0ba989bda635b4e3
I20260808 15:55:45.604072 23888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38: Bootstrap starting.
I20260808 15:55:45.606318 23888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:45.607242 23888 log.cc:824] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:45.609203 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.601211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.30.76.126:44499)
0808 15:55:45.601824 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.601842 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.601876 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:55:45.602508 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.602524 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.602886 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.603219 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:45.603713 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.604713 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.605690 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.606157 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.606347 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.607303 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:45.607310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:45.607327 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:45.607837 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:45.607907 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:45.607937 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:45.608043 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.608245 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.608535 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
I20260808 15:55:45.609345 23888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38: No bootstrap required, opened a new log
I20260808 15:55:45.612709 23888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2913fd13f4f14c57862851285f415a38" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44499 } }
I20260808 15:55:45.612948 23888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:45.613037 23888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2913fd13f4f14c57862851285f415a38, State: Initialized, Role: FOLLOWER
I20260808 15:55:45.613462 23888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2913fd13f4f14c57862851285f415a38" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44499 } }
I20260808 15:55:45.613605 23888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:45.613701 23888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:45.613823 23888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:45.614703 23888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2913fd13f4f14c57862851285f415a38" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44499 } }
I20260808 15:55:45.615041 23888 leader_election.cc:302] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2913fd13f4f14c57862851285f415a38; no voters:
I20260808 15:55:45.615345 23888 leader_election.cc:288] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:45.615489 23895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:45.615784 23895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [term 1 LEADER]: Becoming Leader. State: Replica: 2913fd13f4f14c57862851285f415a38, State: Running, Role: LEADER
I20260808 15:55:45.616089 23895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2913fd13f4f14c57862851285f415a38" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44499 } }
I20260808 15:55:45.616709 23888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:45.617841 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2913fd13f4f14c57862851285f415a38. Latest consensus state: current_term: 1 leader_uuid: "2913fd13f4f14c57862851285f415a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2913fd13f4f14c57862851285f415a38" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44499 } } }
I20260808 15:55:45.617991 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:45.618332 23896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2913fd13f4f14c57862851285f415a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2913fd13f4f14c57862851285f415a38" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44499 } } }
I20260808 15:55:45.618428 23896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:45.618510 23909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:45.621752 23909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:45.627617 23909 catalog_manager.cc:1347] Generated new cluster ID: c8b02a9e244746df810edecccbc4ef2a
I20260808 15:55:45.627700 23909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:45.651468 23909 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:45.652726 23909 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:45.658854 23909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38: Generated new TSK 0
I20260808 15:55:45.659626 23909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:55:45.673732 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44499
--builtin_ntp_servers=127.30.76.84:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:55:45.788447 23922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:45.788700 23922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:45.788738 23922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:45.792696 23922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:45.792747 23922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:45.792799 23922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:45.797402 23922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:45.798442 23922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:45.799842 23922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:45.809662 23930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:45.809662 23932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:45.809715 23929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:45.809991 23922 server_base.cc:1034] running on GCE node
I20260808 15:55:45.810391 23922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:45.811012 23922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:45.812215 23922 hybrid_clock.cc:648] HybridClock initialized: now 1786204545812203 us; error 28 us; skew 500 ppm
I20260808 15:55:45.814110 23922 webserver.cc:460] Webserver started at http://127.30.76.65:46371/ using document root <none> and password file <none>
I20260808 15:55:45.814862 23922 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:45.814983 23922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:45.815198 23922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:45.817746 23922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "d537ec452235dd4e7099a7d47db8d4ba"
server_key_iv: "b8f073cae077ac26af78a3f6432f46eb"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.818393 23922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "d537ec452235dd4e7099a7d47db8d4ba"
server_key_iv: "b8f073cae077ac26af78a3f6432f46eb"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.822050 23922 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:55:45.824457 23938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.825657 23922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:45.825798 23922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2"
format_stamp: "Formatted at 2026-08-08 15:55:45 on dist-test-slave-sb2z"
server_key: "d537ec452235dd4e7099a7d47db8d4ba"
server_key_iv: "b8f073cae077ac26af78a3f6432f46eb"
server_key_version: "encryptionkey@0"
I20260808 15:55:45.825914 23922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:45.855875 23922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:45.856613 23922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:45.856828 23922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:45.857473 23922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:45.858450 23922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:45.858530 23922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.858600 23922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:45.858649 23922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:45.867796 23922 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:40457
I20260808 15:55:45.867842 24103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:40457 every 8 connection(s)
I20260808 15:55:45.868678 23922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:45.871842 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 23922
I20260808 15:55:45.871942 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:55:45.872274 31025 external_mini_cluster.cc:1428] Setting key ff1dc66f081ff7645ab38dfe5792fe90
I20260808 15:55:45.874366 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44499
--builtin_ntp_servers=127.30.76.84:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:55:45.878703 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.870509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:43265 (local address 127.30.76.126:44499)
0808 15:55:45.870651 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:45.870655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:45.871749 (+ 1094us) server_negotiation.cc:407] Connection header received
0808 15:55:45.873466 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:45.873470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:45.873530 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:45.873616 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:45.874489 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.875029 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.876453 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.876818 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.876923 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.878229 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:45.878232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:45.878234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:45.878246 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:45.878265 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:45.878273 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:45.878333 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:45.878525 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 15:55:45.878589 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:55:45.879969 24108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:45.870903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:44499 (local address 127.30.76.65:43265)
0808 15:55:45.871588 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:45.871628 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:45.873248 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:45.873779 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:45.873785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:45.873868 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:45.874335 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.874347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.875177 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.875181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.876290 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:45.876336 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:45.877004 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:45.877010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:45.877259 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:45.877917 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:45.877921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:45.877976 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:45.878060 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:45.878368 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:45.878380 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:45.878389 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 15:55:45.878520 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 15:55:45.878817 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
I20260808 15:55:45.881318 24105 heartbeater.cc:347] Connected to a master server at 127.30.76.126:44499
I20260808 15:55:45.881577 24105 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:45.882094 24105 heartbeater.cc:511] Master 127.30.76.126:44499 requested a full tablet report, sending...
I20260808 15:55:45.883658 23833 ts_manager.cc:194] Registered new tserver with Master: 0ea1f9d4cd4243b6b13307bc3bd3cea2 (127.30.76.65:40457)
I20260808 15:55:45.884831 23833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.65:43265
W20260808 15:55:45.989138 24109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:45.989423 24109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:45.989498 24109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:45.993549 24109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:45.993629 24109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:45.993717 24109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:45.998325 24109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:45.999388 24109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:46.000819 24109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:46.010774 24120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:46.010908 24115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:46.010769 24117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:46.011010 24109 server_base.cc:1034] running on GCE node
I20260808 15:55:46.011598 24109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:46.012305 24109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:46.013487 24109 hybrid_clock.cc:648] HybridClock initialized: now 1786204546013465 us; error 31 us; skew 500 ppm
I20260808 15:55:46.015411 24109 webserver.cc:460] Webserver started at http://127.30.76.66:40379/ using document root <none> and password file <none>
I20260808 15:55:46.016067 24109 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:46.016137 24109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:46.016408 24109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:46.018911 24109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "7fe8224b118c4da1937b3b7cd54e7a5e"
format_stamp: "Formatted at 2026-08-08 15:55:46 on dist-test-slave-sb2z"
server_key: "f99757842c31254b42f88f3ce255c39d"
server_key_iv: "f3813bc7c06808a48054ce41346a34be"
server_key_version: "encryptionkey@0"
I20260808 15:55:46.019629 24109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "7fe8224b118c4da1937b3b7cd54e7a5e"
format_stamp: "Formatted at 2026-08-08 15:55:46 on dist-test-slave-sb2z"
server_key: "f99757842c31254b42f88f3ce255c39d"
server_key_iv: "f3813bc7c06808a48054ce41346a34be"
server_key_version: "encryptionkey@0"
I20260808 15:55:46.023890 24109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:55:46.026521 24126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:46.027940 24109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:55:46.028108 24109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "7fe8224b118c4da1937b3b7cd54e7a5e"
format_stamp: "Formatted at 2026-08-08 15:55:46 on dist-test-slave-sb2z"
server_key: "f99757842c31254b42f88f3ce255c39d"
server_key_iv: "f3813bc7c06808a48054ce41346a34be"
server_key_version: "encryptionkey@0"
I20260808 15:55:46.028255 24109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:46.045408 24109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:46.046221 24109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:46.046463 24109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:46.047138 24109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:46.048259 24109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:46.048345 24109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:46.048425 24109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:46.048489 24109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:46.058085 24109 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:38583
I20260808 15:55:46.058106 24293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:38583 every 8 connection(s)
I20260808 15:55:46.059051 24109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:46.062371 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 24109
I20260808 15:55:46.062541 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:55:46.062867 31025 external_mini_cluster.cc:1428] Setting key d3bd7dae061b0f6168d2a516c87fe9b7
I20260808 15:55:46.065131 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44499
--builtin_ntp_servers=127.30.76.84:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:55:46.068861 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.061095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:39447 (local address 127.30.76.126:44499)
0808 15:55:46.061256 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.061260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.062301 (+ 1041us) server_negotiation.cc:407] Connection header received
0808 15:55:46.064052 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.064055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.064108 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.064225 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:46.065088 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.065701 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.066832 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.067057 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.067177 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.068336 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:46.068339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:46.068342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:46.068362 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:46.068381 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:46.068389 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:46.068473 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.068655 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.068720 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 15:55:46.069654 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.061482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:44499 (local address 127.30.76.66:39447)
0808 15:55:46.062143 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.062182 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.063818 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.064396 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.064402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.064484 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:46.064935 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.064948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.065836 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.065840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.066694 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.066703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.067217 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.067221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.067383 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.068070 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:46.068071 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:46.068109 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:46.068174 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:46.068488 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:46.068499 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:46.068505 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.068644 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.068900 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":128,"threads_started":1}
I20260808 15:55:46.070905 24294 heartbeater.cc:347] Connected to a master server at 127.30.76.126:44499
I20260808 15:55:46.071165 24294 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:46.071779 24294 heartbeater.cc:511] Master 127.30.76.126:44499 requested a full tablet report, sending...
I20260808 15:55:46.072911 23833 ts_manager.cc:194] Registered new tserver with Master: 7fe8224b118c4da1937b3b7cd54e7a5e (127.30.76.66:38583)
I20260808 15:55:46.073800 23833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.66:39447
W20260808 15:55:46.180531 24302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:46.180800 24302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:46.180838 24302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:46.184840 24302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:46.184894 24302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:46.184945 24302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:46.189465 24302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:46.190438 24302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:46.191838 24302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:46.201383 24314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:46.201411 24311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:46.201407 24310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:46.201545 24302 server_base.cc:1034] running on GCE node
I20260808 15:55:46.202080 24302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:46.202623 24302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:46.203784 24302 hybrid_clock.cc:648] HybridClock initialized: now 1786204546203754 us; error 42 us; skew 500 ppm
I20260808 15:55:46.205567 24302 webserver.cc:460] Webserver started at http://127.30.76.67:34779/ using document root <none> and password file <none>
I20260808 15:55:46.206219 24302 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:46.206316 24302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:46.206530 24302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:46.208585 24302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "ee75c47a6fb94477af3ae6d2bc6ee845"
format_stamp: "Formatted at 2026-08-08 15:55:46 on dist-test-slave-sb2z"
server_key: "2d729f11cecc1e01e7511f3b781c3663"
server_key_iv: "a9a642e6c5ab57254a7253514f7e24f4"
server_key_version: "encryptionkey@0"
I20260808 15:55:46.209147 24302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "ee75c47a6fb94477af3ae6d2bc6ee845"
format_stamp: "Formatted at 2026-08-08 15:55:46 on dist-test-slave-sb2z"
server_key: "2d729f11cecc1e01e7511f3b781c3663"
server_key_iv: "a9a642e6c5ab57254a7253514f7e24f4"
server_key_version: "encryptionkey@0"
I20260808 15:55:46.212718 24302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:55:46.215114 24322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:46.216398 24302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:46.216523 24302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "ee75c47a6fb94477af3ae6d2bc6ee845"
format_stamp: "Formatted at 2026-08-08 15:55:46 on dist-test-slave-sb2z"
server_key: "2d729f11cecc1e01e7511f3b781c3663"
server_key_iv: "a9a642e6c5ab57254a7253514f7e24f4"
server_key_version: "encryptionkey@0"
I20260808 15:55:46.216634 24302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:46.240836 24302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:46.241586 24302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:46.241801 24302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:46.242487 24302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:46.243444 24302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:46.243501 24302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:46.243609 24302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:46.243645 24302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:46.252249 24302 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:40509
I20260808 15:55:46.252278 24487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:40509 every 8 connection(s)
I20260808 15:55:46.253167 24302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:46.261833 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.254966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37133 (local address 127.30.76.126:44499)
0808 15:55:46.255116 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.255121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.256243 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 15:55:46.257516 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.257519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.257565 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.257646 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:46.258465 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.258991 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.260162 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.260384 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.260471 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.261430 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:46.261433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:46.261435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:46.261447 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:46.261465 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:46.261473 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:46.261530 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.261677 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.261725 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:55:46.262591 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.255402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:44499 (local address 127.30.76.67:37133)
0808 15:55:46.256079 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.256118 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.257315 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.257792 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.257798 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.257875 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:46.258342 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.258354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.259101 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.259103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.260048 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.260058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.260522 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.260525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.260657 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.261199 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:46.261200 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:46.261236 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:46.261293 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:46.261551 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:46.261558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:46.261562 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.261650 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.261840 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":180,"threads_started":1}
I20260808 15:55:46.262842 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 24302
I20260808 15:55:46.262938 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:55:46.263276 31025 external_mini_cluster.cc:1428] Setting key 0758b53be4e6342bcd7b351152361c49
I20260808 15:55:46.263839 24488 heartbeater.cc:347] Connected to a master server at 127.30.76.126:44499
I20260808 15:55:46.264067 24488 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:46.264654 24488 heartbeater.cc:511] Master 127.30.76.126:44499 requested a full tablet report, sending...
I20260808 15:55:46.265713 23833 ts_manager.cc:194] Registered new tserver with Master: ee75c47a6fb94477af3ae6d2bc6ee845 (127.30.76.67:40509)
I20260808 15:55:46.266229 23833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.67:37133
I20260808 15:55:46.277199 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:46.283215 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.279221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.30.76.126:44499)
0808 15:55:46.279379 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.279383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.279577 (+ 194us) server_negotiation.cc:407] Connection header received
0808 15:55:46.279769 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.279779 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.279820 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.279909 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:46.280211 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.280719 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.281595 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.281819 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.281938 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.282798 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:46.282802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:46.282805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:46.282825 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:46.282842 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:46.282850 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:46.282934 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.283026 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.283079 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:55:46.286182 23833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:46.289331 23833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:46.303191 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.297341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38583 (local address 127.0.0.1:52658)
0808 15:55:46.297694 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.297743 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.297873 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.298527 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.298533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.298569 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:46.298729 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.298736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.300395 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.300398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.301538 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.301549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.302621 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.302624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.302729 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.302770 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.302884 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.303014 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":41,"spinlock_wait_cycles":6656,"thread_start_us":113,"threads_started":1}
I20260808 15:55:46.303437 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.297595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:40457 (local address 127.0.0.1:60296)
0808 15:55:46.298475 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.298489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.298570 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.299087 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.299091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.299109 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:46.299219 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.299225 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.300523 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.300527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.301720 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.301731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.302882 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.302886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.303030 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.303054 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.303126 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.303196 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":6,"spinlock_wait_cycles":5376,"thread_start_us":100,"threads_started":1}
I20260808 15:55:46.303723 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.298168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.30.76.65:40457)
0808 15:55:46.298575 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.298586 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.298617 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:55:46.298713 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.298718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.298900 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.299051 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:46.299373 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.300395 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.301837 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.302543 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.302631 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.302809 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.303332 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.303466 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":104,"threads_started":1}
I20260808 15:55:46.303903 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.298094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:40509 (local address 127.0.0.1:53958)
0808 15:55:46.299078 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.299091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.299263 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.299813 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.299816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.299836 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:46.299913 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.299918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.301295 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.301298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.302195 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.302203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.303433 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.303437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.303560 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.303591 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.303715 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.303790 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":107,"threads_started":1}
I20260808 15:55:46.304035 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.298524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.30.76.67:40509)
0808 15:55:46.298926 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.298937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.299182 (+ 245us) server_negotiation.cc:407] Connection header received
0808 15:55:46.299465 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.299471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.299664 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.299806 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:46.300037 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.301168 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.302322 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.303088 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.303187 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.303373 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.303782 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.303871 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":124,"threads_started":1}
I20260808 15:55:46.303251 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.297498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.30.76.66:38583)
0808 15:55:46.298001 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.298015 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.298039 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:55:46.298123 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.298130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.298342 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.298890 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:46.299041 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.300267 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.301686 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.302485 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.302616 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.302808 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.302908 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.303041 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":160,"threads_started":1}
I20260808 15:55:46.306721 24392 tablet_service.cc:1467] Processing CreateTablet for tablet ace8d15403744be191854ce1dbf61807 (DEFAULT_TABLE table=test-table [id=5360901580c647bcb25d4927aea68223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:46.306738 24007 tablet_service.cc:1467] Processing CreateTablet for tablet ace8d15403744be191854ce1dbf61807 (DEFAULT_TABLE table=test-table [id=5360901580c647bcb25d4927aea68223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:46.307214 24191 tablet_service.cc:1467] Processing CreateTablet for tablet ace8d15403744be191854ce1dbf61807 (DEFAULT_TABLE table=test-table [id=5360901580c647bcb25d4927aea68223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:46.307806 24392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace8d15403744be191854ce1dbf61807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:46.307878 24007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace8d15403744be191854ce1dbf61807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:46.308167 24191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace8d15403744be191854ce1dbf61807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:46.314030 24513 tablet_bootstrap.cc:492] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Bootstrap starting.
I20260808 15:55:46.315595 24516 tablet_bootstrap.cc:492] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: Bootstrap starting.
I20260808 15:55:46.315850 24517 tablet_bootstrap.cc:492] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Bootstrap starting.
I20260808 15:55:46.316706 24513 tablet_bootstrap.cc:654] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:46.317521 24513 log.cc:824] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:46.317837 24516 tablet_bootstrap.cc:654] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:46.317837 24517 tablet_bootstrap.cc:654] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:46.318681 24517 log.cc:824] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:46.318681 24516 log.cc:824] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:46.319674 24513 tablet_bootstrap.cc:492] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: No bootstrap required, opened a new log
I20260808 15:55:46.319900 24513 ts_tablet_manager.cc:1397] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 15:55:46.321153 24517 tablet_bootstrap.cc:492] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: No bootstrap required, opened a new log
I20260808 15:55:46.321189 24516 tablet_bootstrap.cc:492] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: No bootstrap required, opened a new log
I20260808 15:55:46.321344 24517 ts_tablet_manager.cc:1397] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 15:55:46.321424 24516 ts_tablet_manager.cc:1397] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:55:46.324265 24516 raft_consensus.cc:348] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.324306 24513 raft_consensus.cc:348] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.324496 24516 raft_consensus.cc:374] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:46.324543 24516 raft_consensus.cc:729] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ea1f9d4cd4243b6b13307bc3bd3cea2, State: Initialized, Role: FOLLOWER
I20260808 15:55:46.324545 24513 raft_consensus.cc:374] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:46.324599 24513 raft_consensus.cc:729] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee75c47a6fb94477af3ae6d2bc6ee845, State: Initialized, Role: FOLLOWER
I20260808 15:55:46.324991 24516 consensus_queue.cc:260] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.325006 24513 consensus_queue.cc:260] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.325399 24517 raft_consensus.cc:348] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.325616 24517 raft_consensus.cc:374] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:46.325675 24517 raft_consensus.cc:729] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe8224b118c4da1937b3b7cd54e7a5e, State: Initialized, Role: FOLLOWER
I20260808 15:55:46.325973 24513 ts_tablet_manager.cc:1428] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 15:55:46.325992 24516 ts_tablet_manager.cc:1428] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260808 15:55:46.326056 24517 consensus_queue.cc:260] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.326346 24105 heartbeater.cc:503] Master 127.30.76.126:44499 was elected leader, sending a full tablet report...
I20260808 15:55:46.326633 24294 heartbeater.cc:503] Master 127.30.76.126:44499 was elected leader, sending a full tablet report...
I20260808 15:55:46.326921 24517 ts_tablet_manager.cc:1428] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:55:46.327270 24488 heartbeater.cc:503] Master 127.30.76.126:44499 was elected leader, sending a full tablet report...
W20260808 15:55:46.369875 24106 tablet.cc:2369] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:55:46.504156 24489 tablet.cc:2369] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:46.517235 24526 raft_consensus.cc:482] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:46.517442 24526 raft_consensus.cc:504] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.518802 24526 leader_election.cc:288] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee75c47a6fb94477af3ae6d2bc6ee845 (127.30.76.67:40509), 0ea1f9d4cd4243b6b13307bc3bd3cea2 (127.30.76.65:40457)
I20260808 15:55:46.522953 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.519260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:50263 (local address 127.30.76.65:40457)
0808 15:55:46.519443 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.519449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.519569 (+ 120us) server_negotiation.cc:407] Connection header received
0808 15:55:46.519842 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.519846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.519900 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.519996 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:46.520309 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.520804 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.521930 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.522320 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.522460 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.522516 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.522743 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.522812 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 15:55:46.522967 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.519108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:60813 (local address 127.30.76.67:40509)
0808 15:55:46.519239 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.519245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.519352 (+ 107us) server_negotiation.cc:407] Connection header received
0808 15:55:46.519505 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.519507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.519620 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.519732 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:46.520049 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.520624 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.521920 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.522328 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.522463 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.522517 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.522780 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.522846 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:55:46.522967 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.519036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:40509 (local address 127.30.76.66:60813)
0808 15:55:46.519238 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.519255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.519367 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.519747 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.519750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.519765 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:46.519899 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.519906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.520745 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.520751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.521786 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.521796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.522457 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.522460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.522631 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.522655 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.522748 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.522826 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":51}
I20260808 15:55:46.522987 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.519089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:40457 (local address 127.30.76.66:50263)
0808 15:55:46.519446 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.519460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:46.519617 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:46.520012 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:46.520015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:46.520034 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:46.520175 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.520181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.520949 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.520953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.521799 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:46.521808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.522455 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.522459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:46.522587 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.522629 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 15:55:46.522726 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 15:55:46.522800 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":20,"thread_start_us":131,"threads_started":1}
I20260808 15:55:46.523725 24422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8d15403744be191854ce1dbf61807" candidate_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" is_pre_election: true
I20260808 15:55:46.524089 24422 raft_consensus.cc:2454] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fe8224b118c4da1937b3b7cd54e7a5e in term 0.
I20260808 15:55:46.524262 24039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8d15403744be191854ce1dbf61807" candidate_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" is_pre_election: true
I20260808 15:55:46.524547 24039 raft_consensus.cc:2454] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fe8224b118c4da1937b3b7cd54e7a5e in term 0.
I20260808 15:55:46.524647 24129 leader_election.cc:302] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fe8224b118c4da1937b3b7cd54e7a5e, ee75c47a6fb94477af3ae6d2bc6ee845; no voters:
I20260808 15:55:46.524951 24526 raft_consensus.cc:2781] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:46.525041 24526 raft_consensus.cc:482] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:55:46.525075 24526 raft_consensus.cc:3037] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:46.526175 24526 raft_consensus.cc:504] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.526568 24526 leader_election.cc:288] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [CANDIDATE]: Term 1 election: Requested vote from peers ee75c47a6fb94477af3ae6d2bc6ee845 (127.30.76.67:40509), 0ea1f9d4cd4243b6b13307bc3bd3cea2 (127.30.76.65:40457)
I20260808 15:55:46.526856 24422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8d15403744be191854ce1dbf61807" candidate_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845"
I20260808 15:55:46.526986 24422 raft_consensus.cc:3037] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:46.527009 24039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8d15403744be191854ce1dbf61807" candidate_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2"
I20260808 15:55:46.527123 24039 raft_consensus.cc:3037] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:46.528371 24039 raft_consensus.cc:2454] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fe8224b118c4da1937b3b7cd54e7a5e in term 1.
I20260808 15:55:46.528371 24422 raft_consensus.cc:2454] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fe8224b118c4da1937b3b7cd54e7a5e in term 1.
I20260808 15:55:46.528663 24129 leader_election.cc:302] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fe8224b118c4da1937b3b7cd54e7a5e, ee75c47a6fb94477af3ae6d2bc6ee845; no voters:
I20260808 15:55:46.528905 24526 raft_consensus.cc:2781] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:46.529116 24526 raft_consensus.cc:686] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 1 LEADER]: Becoming Leader. State: Replica: 7fe8224b118c4da1937b3b7cd54e7a5e, State: Running, Role: LEADER
I20260808 15:55:46.529444 24526 consensus_queue.cc:237] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } }
I20260808 15:55:46.532824 23830 catalog_manager.cc:5495] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fe8224b118c4da1937b3b7cd54e7a5e (127.30.76.66). New cstate: current_term: 1 leader_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fe8224b118c4da1937b3b7cd54e7a5e" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 } health_report { overall_health: UNKNOWN } } }
W20260808 15:55:46.560511 24296 tablet.cc:2369] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:46.593736 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.590319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.30.76.66:38583)
0808 15:55:46.590460 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:46.590464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:46.590500 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:55:46.590578 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:46.590580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:46.590647 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:46.590730 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:46.590947 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.591438 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.592297 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:46.592504 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:46.592584 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:46.592854 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:46.593269 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:46.593410 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:46.593480 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 15:55:46.593529 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:55:46.599476 24039 raft_consensus.cc:1261] T ace8d15403744be191854ce1dbf61807 P 0ea1f9d4cd4243b6b13307bc3bd3cea2 [term 1 FOLLOWER]: Refusing update from remote peer 7fe8224b118c4da1937b3b7cd54e7a5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:46.599478 24422 raft_consensus.cc:1261] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 1 FOLLOWER]: Refusing update from remote peer 7fe8224b118c4da1937b3b7cd54e7a5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:46.600237 24526 consensus_queue.cc:1035] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ea1f9d4cd4243b6b13307bc3bd3cea2" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:46.600463 24528 consensus_queue.cc:1035] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee75c47a6fb94477af3ae6d2bc6ee845" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:46.609078 24533 mvcc.cc:205] Tried to move back new op lower bound from 7316293822863765504 to 7316293822587604992. Current Snapshot: MvccSnapshot[applied={T|T < 7316293822863765504}]
I20260808 15:55:46.611125 24534 mvcc.cc:205] Tried to move back new op lower bound from 7316293822863765504 to 7316293822587604992. Current Snapshot: MvccSnapshot[applied={T|T < 7316293822863765504}]
I20260808 15:55:46.616230 23830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:46.616366 23830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:46.618727 23830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2913fd13f4f14c57862851285f415a38: Sending DeleteTablet for 3 replicas of tablet ace8d15403744be191854ce1dbf61807
I20260808 15:55:46.619580 24191 tablet_service.cc:1514] Processing DeleteTablet for tablet ace8d15403744be191854ce1dbf61807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:46 UTC) from {username='slave'} at 127.0.0.1:52658
I20260808 15:55:46.619573 24392 tablet_service.cc:1514] Processing DeleteTablet for tablet ace8d15403744be191854ce1dbf61807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:46 UTC) from {username='slave'} at 127.0.0.1:53958
I20260808 15:55:46.619674 24007 tablet_service.cc:1514] Processing DeleteTablet for tablet ace8d15403744be191854ce1dbf61807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:46 UTC) from {username='slave'} at 127.0.0.1:60296
I20260808 15:55:46.620043 24543 tablet_replica.cc:330] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: stopping tablet replica
I20260808 15:55:46.620268 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 23922
I20260808 15:55:46.627015 24543 raft_consensus.cc:2229] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:55:46.627367 24543 raft_consensus.cc:2258] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:46.627748 24544 tablet_replica.cc:330] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: stopping tablet replica
I20260808 15:55:46.628152 24544 raft_consensus.cc:2229] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:46.628554 24544 raft_consensus.cc:2258] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:46.628917 24543 ts_tablet_manager.cc:1905] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:46.631052 24543 ts_tablet_manager.cc:1918] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:46.631177 24543 log.cc:1196] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/ace8d15403744be191854ce1dbf61807
I20260808 15:55:46.631574 24543 ts_tablet_manager.cc:1939] T ace8d15403744be191854ce1dbf61807 P 7fe8224b118c4da1937b3b7cd54e7a5e: Deleting consensus metadata
I20260808 15:55:46.632766 23812 catalog_manager.cc:4841] TS 7fe8224b118c4da1937b3b7cd54e7a5e (127.30.76.66:38583): tablet ace8d15403744be191854ce1dbf61807 (table test-table [id=5360901580c647bcb25d4927aea68223]) successfully deleted
I20260808 15:55:46.633273 24544 ts_tablet_manager.cc:1905] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 15:55:46.635609 23809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:40457 (error 108)
I20260808 15:55:46.636559 24544 ts_tablet_manager.cc:1918] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:46.636667 24544 log.cc:1196] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/ace8d15403744be191854ce1dbf61807
I20260808 15:55:46.636686 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:46.636374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:40457 (local address 127.0.0.1:60298)
0808 15:55:46.636526 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:46.636598 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 15:55:46.636793 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 24109
I20260808 15:55:46.636979 24544 ts_tablet_manager.cc:1939] T ace8d15403744be191854ce1dbf61807 P ee75c47a6fb94477af3ae6d2bc6ee845: Deleting consensus metadata
I20260808 15:55:46.638343 23812 catalog_manager.cc:4841] TS ee75c47a6fb94477af3ae6d2bc6ee845 (127.30.76.67:40509): tablet ace8d15403744be191854ce1dbf61807 (table test-table [id=5360901580c647bcb25d4927aea68223]) successfully deleted
W20260808 15:55:46.641479 23809 catalog_manager.cc:4584] TS 0ea1f9d4cd4243b6b13307bc3bd3cea2 (127.30.76.65:40457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ace8d15403744be191854ce1dbf61807: Network error: Client connection negotiation failed: client connection to 127.30.76.65:40457: connect: Connection refused (error 111)
I20260808 15:55:46.648339 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 24302
I20260808 15:55:46.658303 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 23787
2026-08-08T15:55:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1269 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:46 dist-test-slave-sb2z krb5kdc[24550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:46 dist-test-slave-sb2z krb5kdc[24550](info): set up 2 sockets
Aug 08 15:55:46 dist-test-slave-sb2z krb5kdc[24550](info): commencing operation
krb5kdc: starting...
W20260808 15:55:47.887614 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:55:47 dist-test-slave-sb2z krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:55:48.014603 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41515
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:48.127378 24575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:48.127646 24575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:48.127686 24575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:48.131817 24575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:48.131870 24575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:48.131888 24575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:48.131947 24575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:48.131975 24575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:48.137092 24575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:48.138119 24575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:48.139142 24575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:48.147462 24586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:48.147644 24575 server_base.cc:1034] running on GCE node
W20260808 15:55:48.147537 24583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:48.147459 24584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:48.148317 24575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:48.149400 24575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:48.150607 24575 hybrid_clock.cc:648] HybridClock initialized: now 1786204548150589 us; error 36 us; skew 500 ppm
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:48.153398 24575 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:55:48.154176 24575 webserver.cc:460] Webserver started at http://127.30.76.126:37617/ using document root <none> and password file <none>
I20260808 15:55:48.154659 24575 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:48.154714 24575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:48.154868 24575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:48.156908 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "b23730c5214e478bbe8b4fa136f19395"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "60eb31bdf2c65ed308581210a6a5c40d"
server_key_iv: "f96a0176d415f420ec5250a809855ee3"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.157418 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "b23730c5214e478bbe8b4fa136f19395"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "60eb31bdf2c65ed308581210a6a5c40d"
server_key_iv: "f96a0176d415f420ec5250a809855ee3"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.161041 24575 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:55:48.163259 24599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.164498 24575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:48.164597 24575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "b23730c5214e478bbe8b4fa136f19395"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "60eb31bdf2c65ed308581210a6a5c40d"
server_key_iv: "f96a0176d415f420ec5250a809855ee3"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.164678 24575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:48.180377 24575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:48.183230 24575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:48.183375 24575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:48.190486 24575 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41515
I20260808 15:55:48.190516 24662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41515 every 8 connection(s)
I20260808 15:55:48.191411 24575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:48.194201 24663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:48.199508 24663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395: Bootstrap starting.
I20260808 15:55:48.201567 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 24575
I20260808 15:55:48.201723 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:48.202026 31025 external_mini_cluster.cc:1428] Setting key 4ac11b97d8ec74f92272383a8c8fee27
I20260808 15:55:48.202270 24663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:48.203086 24663 log.cc:824] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:48.205116 24663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395: No bootstrap required, opened a new log
I20260808 15:55:48.208554 24663 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23730c5214e478bbe8b4fa136f19395" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41515 } }
I20260808 15:55:48.208786 24663 raft_consensus.cc:374] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:48.208916 24663 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23730c5214e478bbe8b4fa136f19395, State: Initialized, Role: FOLLOWER
I20260808 15:55:48.209379 24663 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23730c5214e478bbe8b4fa136f19395" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41515 } }
I20260808 15:55:48.209551 24663 raft_consensus.cc:388] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:48.209652 24663 raft_consensus.cc:482] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:48.209786 24663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:48.210840 24663 raft_consensus.cc:504] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23730c5214e478bbe8b4fa136f19395" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41515 } }
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:48.211198 24663 leader_election.cc:302] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b23730c5214e478bbe8b4fa136f19395; no voters:
I20260808 15:55:48.211589 24663 leader_election.cc:288] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:48.211660 24668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:48.211885 24668 raft_consensus.cc:686] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [term 1 LEADER]: Becoming Leader. State: Replica: b23730c5214e478bbe8b4fa136f19395, State: Running, Role: LEADER
I20260808 15:55:48.212776 24668 consensus_queue.cc:237] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23730c5214e478bbe8b4fa136f19395" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41515 } }
I20260808 15:55:48.212945 24663 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:48.214740 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b23730c5214e478bbe8b4fa136f19395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23730c5214e478bbe8b4fa136f19395" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41515 } } }
I20260808 15:55:48.214860 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:48.214776 24672 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b23730c5214e478bbe8b4fa136f19395. Latest consensus state: current_term: 1 leader_uuid: "b23730c5214e478bbe8b4fa136f19395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23730c5214e478bbe8b4fa136f19395" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41515 } } }
I20260808 15:55:48.214911 24672 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:48.215562 24695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:48.217410 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.203224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.30.76.126:41515)
0808 15:55:48.203721 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.203741 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.203786 (+ 45us) server_negotiation.cc:407] Connection header received
0808 15:55:48.204500 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.204517 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.204815 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.205156 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:48.205967 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.206926 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.207892 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.208365 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.208549 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.211634 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:48.211640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:48.211683 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:48.211721 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:48.213532 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.214045 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.214052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.214058 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.214203 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.214575 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.214580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.214581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.215151 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:48.215352 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.216298 (+ 946us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.216645 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"spinlock_wait_cycles":59264,"thread_start_us":147,"threads_started":1}
I20260808 15:55:48.220551 24695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:48.226747 24695 catalog_manager.cc:1347] Generated new cluster ID: 03213630f7d747e896e9074c87cb5be5
I20260808 15:55:48.226823 24695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:48.244798 24695 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:48.245995 24695 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:48.251955 24695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395: Generated new TSK 0
I20260808 15:55:48.252691 24695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:55:48.282497 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41515
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:48.394385 24705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:48.394661 24705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:48.394738 24705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:48.398880 24705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:48.398960 24705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:48.399045 24705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:48.404114 24705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:48.405126 24705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:48.406164 24705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:48.415957 24715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:48.416121 24705 server_base.cc:1034] running on GCE node
W20260808 15:55:48.416198 24717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:48.416018 24720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:48.416697 24705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:48.417339 24705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:48.418537 24705 hybrid_clock.cc:648] HybridClock initialized: now 1786204548418496 us; error 51 us; skew 500 ppm
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:48.421500 24705 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:55:48.422390 24705 webserver.cc:460] Webserver started at http://127.30.76.65:33373/ using document root <none> and password file <none>
I20260808 15:55:48.422914 24705 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:48.422973 24705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:48.423151 24705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:48.425220 24705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "99e72d0a38b848ce8885258b882b3380"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "daf45e945495b430d5230217b5603208"
server_key_iv: "71568ff45184c7cd06c67d75ad23f9de"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.425791 24705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "99e72d0a38b848ce8885258b882b3380"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "daf45e945495b430d5230217b5603208"
server_key_iv: "71568ff45184c7cd06c67d75ad23f9de"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.429550 24705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 15:55:48.431970 24730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.433276 24705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:55:48.433431 24705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "99e72d0a38b848ce8885258b882b3380"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "daf45e945495b430d5230217b5603208"
server_key_iv: "71568ff45184c7cd06c67d75ad23f9de"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.433547 24705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:48.452494 24705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:48.455864 24705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:48.456053 24705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:48.456696 24705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:48.457527 24705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:48.457577 24705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.457653 24705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:48.457685 24705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.467499 24705 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:42843
I20260808 15:55:48.467607 24890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:42843 every 8 connection(s)
I20260808 15:55:48.468490 24705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:48.469341 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 24705
I20260808 15:55:48.469483 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:55:48.469843 31025 external_mini_cluster.cc:1428] Setting key f0de74be7ebf9e1aff09283d9f4a1822
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:55:48.484311 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.470562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:47273 (local address 127.30.76.126:41515)
0808 15:55:48.470720 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.470724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.471906 (+ 1182us) server_negotiation.cc:407] Connection header received
0808 15:55:48.472800 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.472803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.472855 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.472946 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:48.474429 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.474971 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.476874 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.477290 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.477376 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.480549 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:48.480556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:48.480559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:48.480590 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:48.482210 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.482767 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.482770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.482771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.482848 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.483193 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.483195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.483196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.483363 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:48.483435 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.484025 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.484138 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:55:48.485252 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.470963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41515 (local address 127.30.76.65:47273)
0808 15:55:48.471750 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:48.471791 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:48.472587 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:48.473106 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:48.473113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:48.473578 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:48.474244 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.474265 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.475114 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.475121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.476709 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.476734 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.477597 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.477603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.477862 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.478484 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:48.478512 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:48.480358 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:48.482321 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:48.482327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:48.482338 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:48.482590 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:48.482954 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:48.482957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:48.482958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:48.483097 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:48.483455 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:48.483460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:48.483759 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 15:55:48.483967 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 15:55:48.484231 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":153,"threads_started":1}
I20260808 15:55:48.486595 24892 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41515
I20260808 15:55:48.486831 24892 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:48.487340 24892 heartbeater.cc:511] Master 127.30.76.126:41515 requested a full tablet report, sending...
I20260808 15:55:48.488955 24621 ts_manager.cc:194] Registered new tserver with Master: 99e72d0a38b848ce8885258b882b3380 (127.30.76.65:42843)
I20260808 15:55:48.490111 24621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:47273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:55:48.502478 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41515
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:48.613826 24897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:48.614077 24897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:48.614154 24897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:48.618146 24897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:48.618228 24897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:48.618312 24897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:48.623389 24897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:48.624379 24897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:48.625363 24897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:55:48.634617 24897 server_base.cc:1034] running on GCE node
W20260808 15:55:48.634583 24904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:48.634583 24903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:48.634610 24906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:48.635301 24897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:48.635998 24897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:48.637187 24897 hybrid_clock.cc:648] HybridClock initialized: now 1786204548637171 us; error 26 us; skew 500 ppm
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:48.639940 24897 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:55:48.640709 24897 webserver.cc:460] Webserver started at http://127.30.76.66:46097/ using document root <none> and password file <none>
I20260808 15:55:48.641240 24897 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:48.641301 24897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:48.641480 24897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:48.643425 24897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "100b63beeff89b91afec486271ebc81f"
server_key_iv: "bc7f691fcd98380e671efb644864358b"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.644006 24897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "100b63beeff89b91afec486271ebc81f"
server_key_iv: "bc7f691fcd98380e671efb644864358b"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.647513 24897 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:55:48.649842 24914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.651008 24897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:55:48.651155 24897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "100b63beeff89b91afec486271ebc81f"
server_key_iv: "bc7f691fcd98380e671efb644864358b"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.651270 24897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:48.666325 24897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:48.669608 24897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:48.669814 24897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:48.670382 24897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:48.671175 24897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:48.671252 24897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.671320 24897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:48.671373 24897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.679914 24897 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:40773
I20260808 15:55:48.679941 25075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:40773 every 8 connection(s)
I20260808 15:55:48.680764 24897 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:48.689669 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 24897
I20260808 15:55:48.689807 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:55:48.690116 31025 external_mini_cluster.cc:1428] Setting key 3a214994c5d2b1bb85c662485bc1e235
I20260808 15:55:48.694485 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.682577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:58831 (local address 127.30.76.126:41515)
0808 15:55:48.682733 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.682738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.683874 (+ 1136us) server_negotiation.cc:407] Connection header received
0808 15:55:48.684660 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.684663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.684708 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.684782 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:48.685893 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.686398 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.687475 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.687755 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.687836 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.690352 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:48.690356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:48.690359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:48.690385 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:48.692047 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.692532 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.692538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.692543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.692647 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.692972 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.692978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.692982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.693240 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:48.693346 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.693784 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.694360 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:55:48.695025 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.682984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41515 (local address 127.30.76.66:58831)
0808 15:55:48.683733 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:48.683774 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:48.684484 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:48.684940 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:48.684946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:48.685276 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:48.685763 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.685776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.686508 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.686510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.687366 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.687375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.687887 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.687890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.688022 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.688532 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:48.688547 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:48.690149 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:48.692170 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:48.692175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:48.692186 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:48.692426 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:48.692745 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:48.692747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:48.692749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:48.692875 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:48.693346 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:48.693351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:48.693552 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 15:55:48.693698 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 15:55:48.693976 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":117,"threads_started":1}
I20260808 15:55:48.696229 25076 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41515
I20260808 15:55:48.696470 25076 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:48.696936 25076 heartbeater.cc:511] Master 127.30.76.126:41515 requested a full tablet report, sending...
I20260808 15:55:48.698047 24621 ts_manager.cc:194] Registered new tserver with Master: 64fb34d2dd8b4f9fa498c29d3ffea93c (127.30.76.66:40773)
I20260808 15:55:48.698911 24621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:58831
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:55:48.718662 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41515
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:48.834467 25083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:48.834715 25083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:48.834762 25083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:48.838928 25083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:48.838987 25083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:48.839052 25083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:48.844313 25083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:48.845268 25083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:48.846225 25083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:48.855891 25094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:48.855895 25091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:48.856037 25092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:48.856354 25083 server_base.cc:1034] running on GCE node
I20260808 15:55:48.856784 25083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:48.857407 25083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:48.858632 25083 hybrid_clock.cc:648] HybridClock initialized: now 1786204548858615 us; error 31 us; skew 500 ppm
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:48.861436 25083 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:55:48.862327 25083 webserver.cc:460] Webserver started at http://127.30.76.67:38947/ using document root <none> and password file <none>
I20260808 15:55:48.862900 25083 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:48.863003 25083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:48.863250 25083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:48.865305 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "f65d142925f24a4db4b608123027d33a"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "9e4c0303717d749f65f14f0b804ed270"
server_key_iv: "8da75ca76e473c51f67e72f14097ac9c"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.865967 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "f65d142925f24a4db4b608123027d33a"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "9e4c0303717d749f65f14f0b804ed270"
server_key_iv: "8da75ca76e473c51f67e72f14097ac9c"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.869825 25083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:55:48.872274 25102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.873484 25083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:48.873656 25083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "f65d142925f24a4db4b608123027d33a"
format_stamp: "Formatted at 2026-08-08 15:55:48 on dist-test-slave-sb2z"
server_key: "9e4c0303717d749f65f14f0b804ed270"
server_key_iv: "8da75ca76e473c51f67e72f14097ac9c"
server_key_version: "encryptionkey@0"
I20260808 15:55:48.873777 25083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:48.894009 25083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:48.896929 25083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:48.897145 25083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:48.897759 25083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:48.898602 25083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:48.898684 25083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.898756 25083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:48.898811 25083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:48.908006 25083 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:36835
I20260808 15:55:48.908032 25259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:36835 every 8 connection(s)
I20260808 15:55:48.908874 25083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:48.915804 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 25083
I20260808 15:55:48.915952 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:55:48.916287 31025 external_mini_cluster.cc:1428] Setting key b46629295b575eb54fdb6521aa64f85a
Aug 08 15:55:48 dist-test-slave-sb2z krb5kdc[24550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:48.922761 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.910776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:34825 (local address 127.30.76.126:41515)
0808 15:55:48.910919 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.910923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.912024 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 15:55:48.912953 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.912955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.913003 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.913107 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:48.914335 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.914985 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.916492 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.916776 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.916886 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.919620 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:48.919623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:48.919625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:48.919649 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:48.920957 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.921415 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.921418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.921419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.921496 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.921836 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.921840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.921841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.922025 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:48.922115 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.922548 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.922646 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:55:48.923588 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.911170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41515 (local address 127.30.76.67:34825)
0808 15:55:48.911844 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:48.911885 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:48.912759 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:48.913335 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:48.913347 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:48.913732 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:48.914190 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.914203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.915101 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.915103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.916310 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.916323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.916940 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.916944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.917119 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.917843 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:48.917864 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:48.919419 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:48.921066 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:48.921071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:48.921081 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:48.921326 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:48.921577 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:48.921579 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:48.921580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:48.921713 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:48.922124 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:48.922130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:48.922338 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 15:55:48.922486 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 15:55:48.922732 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":107,"threads_started":1}
I20260808 15:55:48.924654 25260 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41515
I20260808 15:55:48.924860 25260 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:48.925282 25260 heartbeater.cc:511] Master 127.30.76.126:41515 requested a full tablet report, sending...
I20260808 15:55:48.926247 24621 ts_manager.cc:194] Registered new tserver with Master: f65d142925f24a4db4b608123027d33a (127.30.76.67:36835)
I20260808 15:55:48.926756 24621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:34825
I20260808 15:55:48.930228 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:48.940163 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.932238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.30.76.126:41515)
0808 15:55:48.932384 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.932389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.932571 (+ 182us) server_negotiation.cc:407] Connection header received
0808 15:55:48.932743 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.932746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.932794 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.932877 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:48.933599 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.934096 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.934966 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.935200 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.935318 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.936963 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:48.936968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:48.936971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:48.936996 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:48.938285 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.938799 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.938802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.938804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.938883 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:48.939229 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:48.939233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:48.939235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:48.939428 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:48.939608 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.939911 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.939999 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:55:48.942996 24621 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:48.946187 24621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:48.960243 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.954313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42843 (local address 127.0.0.1:40756)
0808 15:55:48.955662 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:48.955710 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:48.955844 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:48.956569 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:48.956572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:48.956594 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:48.956717 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.956724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.957759 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.957765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.958936 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.958946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.959873 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.959876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.959950 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.959977 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:55:48.960050 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 15:55:48.960140 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":83,"threads_started":1}
I20260808 15:55:48.960315 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.954490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.30.76.65:42843)
0808 15:55:48.955445 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.955462 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.955986 (+ 524us) server_negotiation.cc:407] Connection header received
0808 15:55:48.956056 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.956060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.956232 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.956391 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:48.956858 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.957617 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.959072 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.959763 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.959879 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.960022 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.960077 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.960172 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":80,"threads_started":1}
I20260808 15:55:48.961272 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.954772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36835 (local address 127.0.0.1:53300)
0808 15:55:48.956078 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:48.956089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:48.956204 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:48.956697 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:48.956701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:48.956713 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:48.956798 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.956805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.958130 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.958133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.959601 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.959613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.960844 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.960848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.960967 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.960990 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:55:48.961068 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:55:48.961135 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"mutex_wait_us":145,"thread_start_us":90,"threads_started":1}
I20260808 15:55:48.961385 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.955183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.30.76.67:36835)
0808 15:55:48.955699 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.955715 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.956195 (+ 480us) server_negotiation.cc:407] Connection header received
0808 15:55:48.956348 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.956355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.956549 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.956702 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:48.956891 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.957950 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.959741 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.960536 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.960649 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.960772 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.961137 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.961236 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":169,"threads_started":1}
I20260808 15:55:48.962538 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.954488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40773 (local address 127.0.0.1:45652)
0808 15:55:48.955732 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:48.955747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:48.955847 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:48.956395 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:48.956400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:48.956423 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:48.956554 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.956573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.958122 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.958125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.959601 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:48.959613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.962151 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.962154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:48.962255 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.962278 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:55:48.962354 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 15:55:48.962417 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":142,"thread_start_us":97,"threads_started":1}
I20260808 15:55:48.962922 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:48.954938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45652 (local address 127.30.76.66:40773)
0808 15:55:48.955911 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:48.955924 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:48.955950 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:55:48.956030 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:48.956035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:48.956232 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:48.956391 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:48.956734 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.957932 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.960493 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:48.961912 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:48.962138 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:48.962272 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:48.962496 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 15:55:48.962585 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":154,"threads_started":1}
I20260808 15:55:48.963584 24792 tablet_service.cc:1467] Processing CreateTablet for tablet 088f773940dc4cb68789ff2a65d491af (DEFAULT_TABLE table=test-table [id=6660c79bc2394888825f7b871c33184c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:48.964026 25173 tablet_service.cc:1467] Processing CreateTablet for tablet 088f773940dc4cb68789ff2a65d491af (DEFAULT_TABLE table=test-table [id=6660c79bc2394888825f7b871c33184c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:48.964691 24792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088f773940dc4cb68789ff2a65d491af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:48.964983 25173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088f773940dc4cb68789ff2a65d491af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:48.966048 24986 tablet_service.cc:1467] Processing CreateTablet for tablet 088f773940dc4cb68789ff2a65d491af (DEFAULT_TABLE table=test-table [id=6660c79bc2394888825f7b871c33184c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:48.967013 24986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088f773940dc4cb68789ff2a65d491af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:48.971356 25278 tablet_bootstrap.cc:492] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Bootstrap starting.
I20260808 15:55:48.972209 25279 tablet_bootstrap.cc:492] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: Bootstrap starting.
I20260808 15:55:48.973907 25280 tablet_bootstrap.cc:492] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Bootstrap starting.
I20260808 15:55:48.974236 25279 tablet_bootstrap.cc:654] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:48.974468 25278 tablet_bootstrap.cc:654] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:48.975718 25279 log.cc:824] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:48.975894 25280 tablet_bootstrap.cc:654] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:48.975878 25278 log.cc:824] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:48.977190 25280 log.cc:824] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:48.977893 25279 tablet_bootstrap.cc:492] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: No bootstrap required, opened a new log
I20260808 15:55:48.978092 25278 tablet_bootstrap.cc:492] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: No bootstrap required, opened a new log
I20260808 15:55:48.978124 25279 ts_tablet_manager.cc:1397] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 15:55:48.978266 25278 ts_tablet_manager.cc:1397] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 15:55:48.979084 25280 tablet_bootstrap.cc:492] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: No bootstrap required, opened a new log
I20260808 15:55:48.979239 25280 ts_tablet_manager.cc:1397] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:55:48.981142 25278 raft_consensus.cc:348] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:48.981389 25278 raft_consensus.cc:374] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:48.981464 25278 raft_consensus.cc:729] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f65d142925f24a4db4b608123027d33a, State: Initialized, Role: FOLLOWER
I20260808 15:55:48.981918 25278 consensus_queue.cc:260] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:48.981930 25279 raft_consensus.cc:348] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:48.982213 25279 raft_consensus.cc:374] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:48.982349 25279 raft_consensus.cc:729] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e72d0a38b848ce8885258b882b3380, State: Initialized, Role: FOLLOWER
I20260808 15:55:48.982571 25260 heartbeater.cc:503] Master 127.30.76.126:41515 was elected leader, sending a full tablet report...
I20260808 15:55:48.982908 25278 ts_tablet_manager.cc:1428] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 15:55:48.982908 25280 raft_consensus.cc:348] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:48.983222 25280 raft_consensus.cc:374] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:48.983278 25280 raft_consensus.cc:729] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64fb34d2dd8b4f9fa498c29d3ffea93c, State: Initialized, Role: FOLLOWER
I20260808 15:55:48.983250 25279 consensus_queue.cc:260] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:48.983697 25280 consensus_queue.cc:260] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:48.983945 24892 heartbeater.cc:503] Master 127.30.76.126:41515 was elected leader, sending a full tablet report...
I20260808 15:55:48.984185 25279 ts_tablet_manager.cc:1428] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260808 15:55:48.986281 25076 heartbeater.cc:503] Master 127.30.76.126:41515 was elected leader, sending a full tablet report...
I20260808 15:55:48.986464 25280 ts_tablet_manager.cc:1428] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260808 15:55:49.159912 25261 tablet.cc:2369] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:49.174605 25286 raft_consensus.cc:482] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:49.174866 25286 raft_consensus.cc:504] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:49.176059 25286 leader_election.cc:288] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99e72d0a38b848ce8885258b882b3380 (127.30.76.65:42843), 64fb34d2dd8b4f9fa498c29d3ffea93c (127.30.76.66:40773)
I20260808 15:55:49.180052 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.176379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:58199 (local address 127.30.76.65:42843)
0808 15:55:49.176515 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:49.176520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:49.176537 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:55:49.176707 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:49.176712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:49.176770 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:49.176885 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:49.177204 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.177836 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.179050 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.179439 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.179598 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.179664 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:49.179849 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 15:55:49.179920 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 15:55:49.180060 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.176546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:40173 (local address 127.30.76.66:40773)
0808 15:55:49.176690 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:49.176694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:49.176838 (+ 144us) server_negotiation.cc:407] Connection header received
0808 15:55:49.176967 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:49.176970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:49.177028 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:49.177131 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:49.177371 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.177977 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.179046 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.179437 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.179593 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.179656 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:49.179887 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 15:55:49.179958 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:55:49.180058 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.176292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42843 (local address 127.30.76.67:58199)
0808 15:55:49.176437 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:49.176454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:49.176551 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:49.176918 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:49.176921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:49.176940 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:49.177046 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.177050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.177943 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.177947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.178909 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.178921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.179584 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.179588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.179710 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.179731 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:55:49.179800 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 15:55:49.179896 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":30}
I20260808 15:55:49.180071 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.176315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:40773 (local address 127.30.76.67:40173)
0808 15:55:49.176736 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:49.176747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:49.176832 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:49.177139 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:49.177144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:49.177163 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:49.177266 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.177272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.178090 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.178093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.178910 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.178920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.179584 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.179588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.179733 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.179750 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:55:49.179854 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 15:55:49.179919 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":27,"thread_start_us":147,"threads_started":1}
I20260808 15:55:49.180737 25013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088f773940dc4cb68789ff2a65d491af" candidate_uuid: "f65d142925f24a4db4b608123027d33a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" is_pre_election: true
I20260808 15:55:49.180747 24818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088f773940dc4cb68789ff2a65d491af" candidate_uuid: "f65d142925f24a4db4b608123027d33a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e72d0a38b848ce8885258b882b3380" is_pre_election: true
I20260808 15:55:49.181053 24818 raft_consensus.cc:2454] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f65d142925f24a4db4b608123027d33a in term 0.
I20260808 15:55:49.181059 25013 raft_consensus.cc:2454] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f65d142925f24a4db4b608123027d33a in term 0.
I20260808 15:55:49.181552 25107 leader_election.cc:302] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99e72d0a38b848ce8885258b882b3380, f65d142925f24a4db4b608123027d33a; no voters:
I20260808 15:55:49.181840 25286 raft_consensus.cc:2781] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:49.181927 25286 raft_consensus.cc:482] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260808 15:55:49.181895 25077 tablet.cc:2369] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:49.181972 25286 raft_consensus.cc:3037] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:49.182982 25286 raft_consensus.cc:504] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:49.183320 25286 leader_election.cc:288] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [CANDIDATE]: Term 1 election: Requested vote from peers 99e72d0a38b848ce8885258b882b3380 (127.30.76.65:42843), 64fb34d2dd8b4f9fa498c29d3ffea93c (127.30.76.66:40773)
I20260808 15:55:49.183660 24818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088f773940dc4cb68789ff2a65d491af" candidate_uuid: "f65d142925f24a4db4b608123027d33a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e72d0a38b848ce8885258b882b3380"
I20260808 15:55:49.183766 25013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088f773940dc4cb68789ff2a65d491af" candidate_uuid: "f65d142925f24a4db4b608123027d33a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c"
I20260808 15:55:49.183794 24818 raft_consensus.cc:3037] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:49.183933 25013 raft_consensus.cc:3037] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:49.184899 24818 raft_consensus.cc:2454] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f65d142925f24a4db4b608123027d33a in term 1.
I20260808 15:55:49.184962 25013 raft_consensus.cc:2454] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f65d142925f24a4db4b608123027d33a in term 1.
I20260808 15:55:49.185235 25107 leader_election.cc:302] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99e72d0a38b848ce8885258b882b3380, f65d142925f24a4db4b608123027d33a; no voters:
I20260808 15:55:49.185406 25286 raft_consensus.cc:2781] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:49.185626 25286 raft_consensus.cc:686] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 1 LEADER]: Becoming Leader. State: Replica: f65d142925f24a4db4b608123027d33a, State: Running, Role: LEADER
I20260808 15:55:49.185949 25286 consensus_queue.cc:237] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } }
I20260808 15:55:49.189172 24619 catalog_manager.cc:5495] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a reported cstate change: term changed from 0 to 1, leader changed from <none> to f65d142925f24a4db4b608123027d33a (127.30.76.67). New cstate: current_term: 1 leader_uuid: "f65d142925f24a4db4b608123027d33a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f65d142925f24a4db4b608123027d33a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36835 } health_report { overall_health: HEALTHY } } }
W20260808 15:55:49.219945 24893 tablet.cc:2369] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:49.253137 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.249882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.30.76.67:36835)
0808 15:55:49.250023 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:49.250027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:49.250040 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:55:49.250113 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:49.250115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:49.250172 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:49.250263 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:49.250501 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.250994 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.251812 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.252017 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.252095 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.252349 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:49.252760 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:49.252866 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:49.252995 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 15:55:49.253031 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:55:49.259033 25013 raft_consensus.cc:1261] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 1 FOLLOWER]: Refusing update from remote peer f65d142925f24a4db4b608123027d33a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:49.259034 24818 raft_consensus.cc:1261] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 1 FOLLOWER]: Refusing update from remote peer f65d142925f24a4db4b608123027d33a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:49.259747 25291 consensus_queue.cc:1035] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [LEADER]: Connected to new peer: Peer: permanent_uuid: "64fb34d2dd8b4f9fa498c29d3ffea93c" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 40773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:49.259966 25286 consensus_queue.cc:1035] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e72d0a38b848ce8885258b882b3380" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:49.272209 25296 mvcc.cc:205] Tried to move back new op lower bound from 7316293833757171712 to 7316293833467985920. Current Snapshot: MvccSnapshot[applied={T|T < 7316293833757171712}]
I20260808 15:55:49.274398 24621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:49.274576 24621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:49.277040 24621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b23730c5214e478bbe8b4fa136f19395: Sending DeleteTablet for 3 replicas of tablet 088f773940dc4cb68789ff2a65d491af
I20260808 15:55:49.277884 24792 tablet_service.cc:1514] Processing DeleteTablet for tablet 088f773940dc4cb68789ff2a65d491af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:49 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:40756
I20260808 15:55:49.277920 25173 tablet_service.cc:1514] Processing DeleteTablet for tablet 088f773940dc4cb68789ff2a65d491af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:49 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:53300
I20260808 15:55:49.277935 24986 tablet_service.cc:1514] Processing DeleteTablet for tablet 088f773940dc4cb68789ff2a65d491af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:49 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:45652
I20260808 15:55:49.278412 25306 tablet_replica.cc:330] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: stopping tablet replica
I20260808 15:55:49.278548 25307 tablet_replica.cc:330] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380: stopping tablet replica
I20260808 15:55:49.278743 25306 raft_consensus.cc:2229] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:55:49.278843 25307 raft_consensus.cc:2229] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:49.279069 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 24705
I20260808 15:55:49.279179 25307 raft_consensus.cc:2258] T 088f773940dc4cb68789ff2a65d491af P 99e72d0a38b848ce8885258b882b3380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:49.282543 25306 raft_consensus.cc:2258] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:49.285344 25306 ts_tablet_manager.cc:1905] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:49.288280 25306 ts_tablet_manager.cc:1918] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:49.288381 25306 log.cc:1196] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/088f773940dc4cb68789ff2a65d491af
I20260808 15:55:49.288713 25308 tablet_replica.cc:330] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: stopping tablet replica
I20260808 15:55:49.288776 25306 ts_tablet_manager.cc:1939] T 088f773940dc4cb68789ff2a65d491af P f65d142925f24a4db4b608123027d33a: Deleting consensus metadata
I20260808 15:55:49.289162 25308 raft_consensus.cc:2229] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:49.289528 25308 raft_consensus.cc:2258] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:49.290150 24602 catalog_manager.cc:4841] TS f65d142925f24a4db4b608123027d33a (127.30.76.67:36835): tablet 088f773940dc4cb68789ff2a65d491af (table test-table [id=6660c79bc2394888825f7b871c33184c]) successfully deleted
I20260808 15:55:49.290992 25308 ts_tablet_manager.cc:1905] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:49.293161 25308 ts_tablet_manager.cc:1918] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:49.293282 25308 log.cc:1196] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/088f773940dc4cb68789ff2a65d491af
I20260808 15:55:49.293672 25308 ts_tablet_manager.cc:1939] T 088f773940dc4cb68789ff2a65d491af P 64fb34d2dd8b4f9fa498c29d3ffea93c: Deleting consensus metadata
I20260808 15:55:49.294929 24600 catalog_manager.cc:4841] TS 64fb34d2dd8b4f9fa498c29d3ffea93c (127.30.76.66:40773): tablet 088f773940dc4cb68789ff2a65d491af (table test-table [id=6660c79bc2394888825f7b871c33184c]) successfully deleted
W20260808 15:55:49.298646 24606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:42843 (error 108)
I20260808 15:55:49.298746 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 24897
I20260808 15:55:49.306696 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.306354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:42843 (local address 127.0.0.1:40762)
0808 15:55:49.306517 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:49.306602 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 15:55:49.306978 24606 catalog_manager.cc:4584] TS 99e72d0a38b848ce8885258b882b3380 (127.30.76.65:42843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 088f773940dc4cb68789ff2a65d491af: Network error: Client connection negotiation failed: client connection to 127.30.76.65:42843: connect: Connection refused (error 111)
I20260808 15:55:49.311239 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 25083
I20260808 15:55:49.318534 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 24575
2026-08-08T15:55:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2664 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T15:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:49Z Disabled control of system clock
I20260808 15:55:49.356335 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45721
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:55:49.466610 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:49.466874 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:49.466908 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:49.471076 25316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:49.471134 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:49.471150 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:49.471167 25316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:49.471182 25316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:49.475898 25316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:49.476965 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:49.478343 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:49.486717 25326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:49.486774 25323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:49.487025 25316 server_base.cc:1034] running on GCE node
W20260808 15:55:49.486778 25331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:49.487802 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:49.488886 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:49.490056 25316 hybrid_clock.cc:648] HybridClock initialized: now 1786204549490027 us; error 42 us; skew 500 ppm
I20260808 15:55:49.491948 25316 webserver.cc:460] Webserver started at http://127.30.76.126:35731/ using document root <none> and password file <none>
I20260808 15:55:49.492522 25316 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:49.492586 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:49.492851 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:49.494938 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "04a1fec43b07d4bbb4412f206eef5661"
server_key_iv: "71c2a199d23ff1bc44b1885b2af145d6"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.495579 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "04a1fec43b07d4bbb4412f206eef5661"
server_key_iv: "71c2a199d23ff1bc44b1885b2af145d6"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.499330 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:55:49.501632 25337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.502837 25316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:49.502985 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "04a1fec43b07d4bbb4412f206eef5661"
server_key_iv: "71c2a199d23ff1bc44b1885b2af145d6"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.503100 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:49.521683 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:49.522418 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:49.522637 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:49.529570 25316 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45721
I20260808 15:55:49.529585 25404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45721 every 8 connection(s)
I20260808 15:55:49.530493 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:49.531980 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 25316
I20260808 15:55:49.532130 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:49.532444 31025 external_mini_cluster.cc:1428] Setting key 2e8bd4ee112dfe919e6b050a44c57c4b
I20260808 15:55:49.533892 25405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:49.539342 25405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5: Bootstrap starting.
I20260808 15:55:49.541668 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.533944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.30.76.126:45721)
0808 15:55:49.534597 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:49.534611 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:49.534649 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:55:49.535289 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:49.535305 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:49.535763 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:49.536097 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:49.536523 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.537467 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.538374 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.538835 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.539011 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.539979 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:49.539984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:49.539996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:49.540365 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:49.540417 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:49.540436 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:49.540554 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:49.540710 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 15:55:49.540992 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":164,"threads_started":1}
I20260808 15:55:49.541679 25405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:49.542655 25405 log.cc:824] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:49.544718 25405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5: No bootstrap required, opened a new log
I20260808 15:55:49.548283 25405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45721 } }
I20260808 15:55:49.548493 25405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:49.548537 25405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9ffb79d6e3455f84f19f8b81ae33d5, State: Initialized, Role: FOLLOWER
I20260808 15:55:49.548944 25405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45721 } }
I20260808 15:55:49.549063 25405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:49.549113 25405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:49.549176 25405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:49.549980 25405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45721 } }
I20260808 15:55:49.550261 25405 leader_election.cc:302] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c9ffb79d6e3455f84f19f8b81ae33d5; no voters:
I20260808 15:55:49.550509 25405 leader_election.cc:288] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:49.550676 25410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:49.550925 25410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [term 1 LEADER]: Becoming Leader. State: Replica: 9c9ffb79d6e3455f84f19f8b81ae33d5, State: Running, Role: LEADER
I20260808 15:55:49.551229 25410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45721 } }
I20260808 15:55:49.551788 25405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:49.553009 25412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c9ffb79d6e3455f84f19f8b81ae33d5. Latest consensus state: current_term: 1 leader_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45721 } } }
I20260808 15:55:49.553009 25411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c9ffb79d6e3455f84f19f8b81ae33d5" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45721 } } }
I20260808 15:55:49.553257 25411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:49.553284 25412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:49.554116 25420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:49.557554 25420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:49.562776 25420 catalog_manager.cc:1347] Generated new cluster ID: e018539332d04035b292f751b316616e
I20260808 15:55:49.562839 25420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:49.580778 25420 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:49.581996 25420 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:49.590610 25420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5: Generated new TSK 0
I20260808 15:55:49.591379 25420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:55:49.600469 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45721
--builtin_ntp_servers=127.30.76.84:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:55:49.715039 25431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:49.715315 25431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:49.715411 25431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:49.719419 25431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:49.719525 25431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:49.719632 25431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:49.724381 25431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:49.725438 25431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:49.726903 25431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:49.736476 25443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:49.736492 25442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:49.736492 25445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:49.736882 25431 server_base.cc:1034] running on GCE node
I20260808 15:55:49.737389 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:49.738051 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:49.739264 25431 hybrid_clock.cc:648] HybridClock initialized: now 1786204549739247 us; error 27 us; skew 500 ppm
I20260808 15:55:49.741216 25431 webserver.cc:460] Webserver started at http://127.30.76.65:34893/ using document root <none> and password file <none>
I20260808 15:55:49.741833 25431 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:49.741936 25431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:49.742161 25431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:49.744356 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "b098a6fe10494d3591299acd185a08d7"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "9c97f23c417893cd140aac14f6747a3c"
server_key_iv: "dee48803472c37520c7f4f5feba2db26"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.744951 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "b098a6fe10494d3591299acd185a08d7"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "9c97f23c417893cd140aac14f6747a3c"
server_key_iv: "dee48803472c37520c7f4f5feba2db26"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.748816 25431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:55:49.751258 25453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.752580 25431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:49.752728 25431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "b098a6fe10494d3591299acd185a08d7"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "9c97f23c417893cd140aac14f6747a3c"
server_key_iv: "dee48803472c37520c7f4f5feba2db26"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.752862 25431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:49.788792 25431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:49.789526 25431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:49.789755 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:49.790392 25431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:49.791375 25431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:49.791456 25431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.791570 25431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:49.791635 25431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.801208 25431 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:39953
I20260808 15:55:49.801241 25604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:39953 every 8 connection(s)
I20260808 15:55:49.802148 25431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:49.808290 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 25431
I20260808 15:55:49.808399 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:55:49.808781 31025 external_mini_cluster.cc:1428] Setting key b6bdd8166b52b9e73e20863edc5e5016
I20260808 15:55:49.810853 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45721
--builtin_ntp_servers=127.30.76.84:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:55:49.811968 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.804123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:50663 (local address 127.30.76.126:45721)
0808 15:55:49.804287 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:49.804291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:49.805241 (+ 950us) server_negotiation.cc:407] Connection header received
0808 15:55:49.806760 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:49.806762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:49.806815 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:49.806893 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:49.807896 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.808508 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.809856 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.810118 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.810242 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.811417 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:49.811420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:49.811423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:49.811436 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:49.811461 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:49.811471 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:49.811576 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:49.811750 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 15:55:49.811817 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:55:49.812772 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.804362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45721 (local address 127.30.76.65:50663)
0808 15:55:49.805074 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:49.805117 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:49.806562 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:49.807058 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:49.807064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:49.807145 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:49.807764 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.807779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.808625 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.808629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.809708 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.809719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.810296 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.810300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.810465 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.811150 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:49.811157 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:49.811197 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:49.811261 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:49.811580 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:49.811593 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:49.811600 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:55:49.811736 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 15:55:49.812003 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":117,"threads_started":1}
I20260808 15:55:49.814195 25605 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45721
I20260808 15:55:49.814430 25605 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:49.814924 25605 heartbeater.cc:511] Master 127.30.76.126:45721 requested a full tablet report, sending...
I20260808 15:55:49.816617 25361 ts_manager.cc:194] Registered new tserver with Master: b098a6fe10494d3591299acd185a08d7 (127.30.76.65:39953)
I20260808 15:55:49.818354 25361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.65:50663
W20260808 15:55:49.924610 25609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:49.924852 25609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:49.924889 25609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:49.928733 25609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:49.928779 25609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:49.928829 25609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:49.933604 25609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:49.934552 25609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:49.935945 25609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:49.945377 25618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:49.945413 25615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:49.945600 25609 server_base.cc:1034] running on GCE node
W20260808 15:55:49.945600 25616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:49.946105 25609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:49.946717 25609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:49.947906 25609 hybrid_clock.cc:648] HybridClock initialized: now 1786204549947887 us; error 28 us; skew 500 ppm
I20260808 15:55:49.949512 25609 webserver.cc:460] Webserver started at http://127.30.76.66:42033/ using document root <none> and password file <none>
I20260808 15:55:49.950026 25609 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:49.950083 25609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:49.950238 25609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:49.952251 25609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "2c977154e8fd435aa1486eb1a5f51f63"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "9c02c0cd54c7101f9e2998499801ee7b"
server_key_iv: "ccedaecc8aaf5eb5a73fceda7683d04b"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.952832 25609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "2c977154e8fd435aa1486eb1a5f51f63"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "9c02c0cd54c7101f9e2998499801ee7b"
server_key_iv: "ccedaecc8aaf5eb5a73fceda7683d04b"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.956425 25609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:55:49.958716 25627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.959962 25609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:55:49.960083 25609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "2c977154e8fd435aa1486eb1a5f51f63"
format_stamp: "Formatted at 2026-08-08 15:55:49 on dist-test-slave-sb2z"
server_key: "9c02c0cd54c7101f9e2998499801ee7b"
server_key_iv: "ccedaecc8aaf5eb5a73fceda7683d04b"
server_key_version: "encryptionkey@0"
I20260808 15:55:49.960183 25609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:49.972746 25609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:49.973315 25609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:49.973500 25609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:49.974041 25609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:49.974908 25609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:49.974954 25609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.975021 25609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:49.975055 25609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:49.983496 25609 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:38107
I20260808 15:55:49.983570 25791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:38107 every 8 connection(s)
I20260808 15:55:49.984431 25609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:49.988499 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 25609
I20260808 15:55:49.988662 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:55:49.989014 31025 external_mini_cluster.cc:1428] Setting key b628eae77eed3a35b403b263b22bc451
I20260808 15:55:49.991226 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45721
--builtin_ntp_servers=127.30.76.84:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:55:49.994774 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.986185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56367 (local address 127.30.76.126:45721)
0808 15:55:49.986335 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:49.986338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:49.987316 (+ 978us) server_negotiation.cc:407] Connection header received
0808 15:55:49.988927 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:49.988930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:49.988979 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:49.989054 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:49.990094 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.990696 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.992599 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.992812 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.992894 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.994209 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:49.994215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:49.994219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:49.994235 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:49.994257 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:49.994268 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:49.994350 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:49.994493 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 15:55:49.994566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:55:49.995486 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:49.986549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45721 (local address 127.30.76.66:56367)
0808 15:55:49.987166 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:49.987204 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:49.988697 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:49.989282 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:49.989292 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:49.989394 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:49.989953 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.989972 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.990834 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.990838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.992468 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:49.992484 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:49.992979 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:49.992985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:49.993244 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:49.993924 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:49.993926 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:49.993964 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:49.994024 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:49.994355 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:49.994362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:49.994366 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:55:49.994458 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:55:49.994668 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":100,"threads_started":1}
I20260808 15:55:49.996719 25792 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45721
I20260808 15:55:49.996939 25792 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:49.997443 25792 heartbeater.cc:511] Master 127.30.76.126:45721 requested a full tablet report, sending...
I20260808 15:55:49.998461 25361 ts_manager.cc:194] Registered new tserver with Master: 2c977154e8fd435aa1486eb1a5f51f63 (127.30.76.66:38107)
I20260808 15:55:49.999040 25361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.66:56367
W20260808 15:55:50.105851 25795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:50.106107 25795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:50.106191 25795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:50.110249 25795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:50.110299 25795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:50.110404 25795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:50.115163 25795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:45721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:50.116216 25795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:50.117559 25795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:55:50.127029 25795 server_base.cc:1034] running on GCE node
W20260808 15:55:50.127003 25807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:50.127146 25803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:50.127305 25804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:50.127746 25795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:50.128353 25795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:50.129513 25795 hybrid_clock.cc:648] HybridClock initialized: now 1786204550129479 us; error 44 us; skew 500 ppm
I20260808 15:55:50.131234 25795 webserver.cc:460] Webserver started at http://127.30.76.67:41603/ using document root <none> and password file <none>
I20260808 15:55:50.131845 25795 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:50.131907 25795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:50.132138 25795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:50.134114 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "6160ba053ad546d5b72b0ea7abe7ed80"
format_stamp: "Formatted at 2026-08-08 15:55:50 on dist-test-slave-sb2z"
server_key: "4eee14e43e6ebfc7742efa82b3b69299"
server_key_iv: "91d992b574e0951e02b59846b868118d"
server_key_version: "encryptionkey@0"
I20260808 15:55:50.134678 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "6160ba053ad546d5b72b0ea7abe7ed80"
format_stamp: "Formatted at 2026-08-08 15:55:50 on dist-test-slave-sb2z"
server_key: "4eee14e43e6ebfc7742efa82b3b69299"
server_key_iv: "91d992b574e0951e02b59846b868118d"
server_key_version: "encryptionkey@0"
I20260808 15:55:50.138284 25795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 15:55:50.140627 25813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:50.141793 25795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:55:50.141912 25795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "6160ba053ad546d5b72b0ea7abe7ed80"
format_stamp: "Formatted at 2026-08-08 15:55:50 on dist-test-slave-sb2z"
server_key: "4eee14e43e6ebfc7742efa82b3b69299"
server_key_iv: "91d992b574e0951e02b59846b868118d"
server_key_version: "encryptionkey@0"
I20260808 15:55:50.142009 25795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:50.159133 25795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:50.159829 25795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:50.160041 25795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:50.160614 25795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:50.161545 25795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:50.161592 25795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:50.161671 25795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:50.161715 25795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:50.170253 25795 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:39509
I20260808 15:55:50.170284 25968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:39509 every 8 connection(s)
I20260808 15:55:50.171103 25795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:50.178875 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 25795
I20260808 15:55:50.178998 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:55:50.179354 31025 external_mini_cluster.cc:1428] Setting key 64c43ece144495ed5e04d0a8999cb8b3
I20260808 15:55:50.180006 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.172915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:44539 (local address 127.30.76.126:45721)
0808 15:55:50.173048 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.173051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.174001 (+ 950us) server_negotiation.cc:407] Connection header received
0808 15:55:50.175276 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.175278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.175325 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.175399 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:50.176287 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.176804 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.177862 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.178063 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.178154 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.179482 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:50.179486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:50.179488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:50.179504 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:50.179542 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:50.179553 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:50.179630 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.179798 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.179858 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:55:50.180999 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.173233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:45721 (local address 127.30.76.67:44539)
0808 15:55:50.173844 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.173882 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:50.175094 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:50.175619 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:50.175634 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:50.175713 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:50.176158 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.176170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.176912 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.176914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.177757 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.177766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.178226 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.178231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.178453 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.179175 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:50.179178 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:50.179231 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:50.179303 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:50.179644 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:50.179651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:50.179655 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:55:50.179788 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 15:55:50.180043 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
I20260808 15:55:50.181991 25969 heartbeater.cc:347] Connected to a master server at 127.30.76.126:45721
I20260808 15:55:50.182224 25969 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:50.182719 25969 heartbeater.cc:511] Master 127.30.76.126:45721 requested a full tablet report, sending...
I20260808 15:55:50.183748 25361 ts_manager.cc:194] Registered new tserver with Master: 6160ba053ad546d5b72b0ea7abe7ed80 (127.30.76.67:39509)
I20260808 15:55:50.184266 25361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.67:44539
I20260808 15:55:50.193503 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:50.199503 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.195441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44066 (local address 127.30.76.126:45721)
0808 15:55:50.195617 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.195622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.195776 (+ 154us) server_negotiation.cc:407] Connection header received
0808 15:55:50.195925 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.195928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.195977 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.196072 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:50.196355 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.196850 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.197708 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.197925 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.198045 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.198986 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:50.198994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:50.198996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:50.199008 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:50.199024 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:50.199032 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:50.199176 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.199327 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.199363 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 15:55:50.202282 25361 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:50.205344 25361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:50.219911 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.214707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.30.76.67:39509)
0808 15:55:50.215374 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.215386 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.215412 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:55:50.215491 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.215496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.215720 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.215875 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:50.216178 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.216937 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.218624 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.219208 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.219344 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.219538 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.219657 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.219751 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":101,"threads_started":1}
I20260808 15:55:50.219905 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.213972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:39509 (local address 127.0.0.1:48820)
0808 15:55:50.215111 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.215125 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:50.215207 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:50.215957 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:50.215960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:50.215975 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:50.216072 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.216078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.217072 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.217076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.218482 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.218496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.219344 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.219347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.219452 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.219486 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 15:55:50.219621 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:55:50.219743 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260808 15:55:50.220206 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.213845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:39953 (local address 127.0.0.1:55642)
0808 15:55:50.214705 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.214758 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:50.214890 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:50.215787 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:50.215793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:50.215816 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:50.215957 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.215964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.217468 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.217471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.218685 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.218696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.219794 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.219797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.219894 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.219919 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:55:50.219965 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 15:55:50.220048 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":43,"thread_start_us":93,"threads_started":1}
I20260808 15:55:50.221035 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.214002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55642 (local address 127.30.76.65:39953)
0808 15:55:50.215209 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.215221 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.215247 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:55:50.215331 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.215336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.215571 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.215790 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:50.216090 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.217323 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.218825 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.219661 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.220512 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.220681 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.220736 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.220829 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1084,"thread_start_us":166,"threads_started":1}
I20260808 15:55:50.221338 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.214671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.30.76.66:38107)
0808 15:55:50.215166 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.215179 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.215469 (+ 290us) server_negotiation.cc:407] Connection header received
0808 15:55:50.215707 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.215712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.215875 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.216004 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:50.216296 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.217396 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.219453 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.220425 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.220506 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.220683 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.221073 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.221182 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":127,"threads_started":1}
I20260808 15:55:50.221774 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.214239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:38107 (local address 127.0.0.1:51254)
0808 15:55:50.215386 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.215398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:50.215557 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:50.216041 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:50.216044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:50.216062 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:50.216162 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.216168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.217510 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.217512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.219307 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.219318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.220785 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.220788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.220871 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.220893 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:55:50.221545 (+ 652us) client_negotiation.cc:240] Negotiation successful
0808 15:55:50.221622 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"mutex_wait_us":45,"thread_start_us":97,"threads_started":1}
I20260808 15:55:50.223107 25888 tablet_service.cc:1467] Processing CreateTablet for tablet 862412c9f0b645a99f0b8399e898e48e (DEFAULT_TABLE table=test-table [id=7c9ea865c6cd4ef2bf95359f3a9812e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:50.224107 25888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862412c9f0b645a99f0b8399e898e48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:50.223781 25516 tablet_service.cc:1467] Processing CreateTablet for tablet 862412c9f0b645a99f0b8399e898e48e (DEFAULT_TABLE table=test-table [id=7c9ea865c6cd4ef2bf95359f3a9812e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:50.224375 25699 tablet_service.cc:1467] Processing CreateTablet for tablet 862412c9f0b645a99f0b8399e898e48e (DEFAULT_TABLE table=test-table [id=7c9ea865c6cd4ef2bf95359f3a9812e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:50.224817 25516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862412c9f0b645a99f0b8399e898e48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:50.225327 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862412c9f0b645a99f0b8399e898e48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:50.231236 26000 tablet_bootstrap.cc:492] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Bootstrap starting.
I20260808 15:55:50.231824 26002 tablet_bootstrap.cc:492] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: Bootstrap starting.
I20260808 15:55:50.232681 26004 tablet_bootstrap.cc:492] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Bootstrap starting.
I20260808 15:55:50.233727 26002 tablet_bootstrap.cc:654] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:50.233727 26000 tablet_bootstrap.cc:654] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:50.234390 26004 tablet_bootstrap.cc:654] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:50.234622 26002 log.cc:824] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:50.234645 26000 log.cc:824] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:50.235209 26004 log.cc:824] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:50.236796 26002 tablet_bootstrap.cc:492] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: No bootstrap required, opened a new log
I20260808 15:55:50.236796 26000 tablet_bootstrap.cc:492] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: No bootstrap required, opened a new log
I20260808 15:55:50.236989 26000 ts_tablet_manager.cc:1397] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:55:50.236989 26002 ts_tablet_manager.cc:1397] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:55:50.237285 26004 tablet_bootstrap.cc:492] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: No bootstrap required, opened a new log
I20260808 15:55:50.237489 26004 ts_tablet_manager.cc:1397] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 15:55:50.240557 26004 raft_consensus.cc:348] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.240806 26004 raft_consensus.cc:374] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:50.240861 26004 raft_consensus.cc:729] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c977154e8fd435aa1486eb1a5f51f63, State: Initialized, Role: FOLLOWER
I20260808 15:55:50.241338 26004 consensus_queue.cc:260] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.241318 26000 raft_consensus.cc:348] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.241318 26002 raft_consensus.cc:348] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.241567 26002 raft_consensus.cc:374] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:50.241567 26000 raft_consensus.cc:374] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:50.241616 26002 raft_consensus.cc:729] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b098a6fe10494d3591299acd185a08d7, State: Initialized, Role: FOLLOWER
I20260808 15:55:50.241616 26000 raft_consensus.cc:729] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6160ba053ad546d5b72b0ea7abe7ed80, State: Initialized, Role: FOLLOWER
I20260808 15:55:50.242033 25792 heartbeater.cc:503] Master 127.30.76.126:45721 was elected leader, sending a full tablet report...
I20260808 15:55:50.242043 26000 consensus_queue.cc:260] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.242043 26002 consensus_queue.cc:260] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.242858 25605 heartbeater.cc:503] Master 127.30.76.126:45721 was elected leader, sending a full tablet report...
I20260808 15:55:50.242904 25969 heartbeater.cc:503] Master 127.30.76.126:45721 was elected leader, sending a full tablet report...
I20260808 15:55:50.243012 26000 ts_tablet_manager.cc:1428] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:55:50.243549 26002 ts_tablet_manager.cc:1428] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 15:55:50.245014 26004 ts_tablet_manager.cc:1428] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260808 15:55:50.291009 26008 raft_consensus.cc:482] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:50.291260 26008 raft_consensus.cc:504] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.292466 26008 leader_election.cc:288] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b098a6fe10494d3591299acd185a08d7 (127.30.76.65:39953), 6160ba053ad546d5b72b0ea7abe7ed80 (127.30.76.67:39509)
I20260808 15:55:50.297561 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.292772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:36763 (local address 127.30.76.65:39953)
0808 15:55:50.292905 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.292910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.292927 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:55:50.293114 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.293117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.293191 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.293280 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:50.293635 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.294422 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.296290 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.296893 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.297022 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.297069 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.297362 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.297436 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:55:50.297560 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.292865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:35335 (local address 127.30.76.67:39509)
0808 15:55:50.293050 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.293056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.293201 (+ 145us) server_negotiation.cc:407] Connection header received
0808 15:55:50.293263 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.293269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.293313 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.293387 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:50.293669 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.294422 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.296290 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.296893 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.297020 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.297069 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.297362 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.297436 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 15:55:50.297580 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.292667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:39953 (local address 127.30.76.66:36763)
0808 15:55:50.292837 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.292855 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:50.292965 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:50.293318 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:50.293320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:50.293334 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:50.293477 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.293485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.294653 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.294657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.296163 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.296170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.297038 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.297041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.297189 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.297250 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 15:55:50.297320 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 15:55:50.297410 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":205}
I20260808 15:55:50.297590 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.292700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:39509 (local address 127.30.76.66:35335)
0808 15:55:50.293107 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.293118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:50.293195 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:50.293408 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:50.293410 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:50.293422 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:50.293549 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.293559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.294653 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.294657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.296158 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:50.296170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.297029 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.297032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:50.297186 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.297242 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 15:55:50.297320 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:55:50.297409 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":179,"thread_start_us":132,"threads_started":1}
I20260808 15:55:50.298386 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862412c9f0b645a99f0b8399e898e48e" candidate_uuid: "2c977154e8fd435aa1486eb1a5f51f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" is_pre_election: true
I20260808 15:55:50.298380 25541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862412c9f0b645a99f0b8399e898e48e" candidate_uuid: "2c977154e8fd435aa1486eb1a5f51f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b098a6fe10494d3591299acd185a08d7" is_pre_election: true
I20260808 15:55:50.298686 25541 raft_consensus.cc:2454] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c977154e8fd435aa1486eb1a5f51f63 in term 0.
I20260808 15:55:50.298686 25915 raft_consensus.cc:2454] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c977154e8fd435aa1486eb1a5f51f63 in term 0.
I20260808 15:55:50.299237 25633 leader_election.cc:302] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c977154e8fd435aa1486eb1a5f51f63, 6160ba053ad546d5b72b0ea7abe7ed80, b098a6fe10494d3591299acd185a08d7; no voters:
I20260808 15:55:50.299582 26008 raft_consensus.cc:2781] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:50.299698 26008 raft_consensus.cc:482] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:55:50.299760 26008 raft_consensus.cc:3037] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:50.300773 26008 raft_consensus.cc:504] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.301146 26008 leader_election.cc:288] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [CANDIDATE]: Term 1 election: Requested vote from peers b098a6fe10494d3591299acd185a08d7 (127.30.76.65:39953), 6160ba053ad546d5b72b0ea7abe7ed80 (127.30.76.67:39509)
I20260808 15:55:50.301505 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862412c9f0b645a99f0b8399e898e48e" candidate_uuid: "2c977154e8fd435aa1486eb1a5f51f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6160ba053ad546d5b72b0ea7abe7ed80"
I20260808 15:55:50.301505 25541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862412c9f0b645a99f0b8399e898e48e" candidate_uuid: "2c977154e8fd435aa1486eb1a5f51f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b098a6fe10494d3591299acd185a08d7"
I20260808 15:55:50.301630 25915 raft_consensus.cc:3037] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:50.301630 25541 raft_consensus.cc:3037] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:50.302884 25541 raft_consensus.cc:2454] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c977154e8fd435aa1486eb1a5f51f63 in term 1.
I20260808 15:55:50.302884 25915 raft_consensus.cc:2454] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c977154e8fd435aa1486eb1a5f51f63 in term 1.
I20260808 15:55:50.303222 25632 leader_election.cc:302] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c977154e8fd435aa1486eb1a5f51f63, 6160ba053ad546d5b72b0ea7abe7ed80, b098a6fe10494d3591299acd185a08d7; no voters:
W20260808 15:55:50.303349 25606 tablet.cc:2369] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:50.303421 26008 raft_consensus.cc:2781] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:50.303714 26008 raft_consensus.cc:686] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 1 LEADER]: Becoming Leader. State: Replica: 2c977154e8fd435aa1486eb1a5f51f63, State: Running, Role: LEADER
I20260808 15:55:50.304075 26008 consensus_queue.cc:237] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } }
I20260808 15:55:50.307379 25361 catalog_manager.cc:5495] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c977154e8fd435aa1486eb1a5f51f63 (127.30.76.66). New cstate: current_term: 1 leader_uuid: "2c977154e8fd435aa1486eb1a5f51f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c977154e8fd435aa1486eb1a5f51f63" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 38107 } health_report { overall_health: HEALTHY } } }
I20260808 15:55:50.318289 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.314980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.30.76.66:38107)
0808 15:55:50.315096 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:50.315100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:50.315113 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:55:50.315185 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:50.315187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:50.315234 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:50.315332 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:50.315671 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.316138 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.317041 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:50.317244 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:50.317322 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:50.317585 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:50.317957 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:50.318062 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:50.318144 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 15:55:50.318173 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:55:50.323689 25915 raft_consensus.cc:1261] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 1 FOLLOWER]: Refusing update from remote peer 2c977154e8fd435aa1486eb1a5f51f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:50.323695 25541 raft_consensus.cc:1261] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7 [term 1 FOLLOWER]: Refusing update from remote peer 2c977154e8fd435aa1486eb1a5f51f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:50.324452 26013 consensus_queue.cc:1035] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6160ba053ad546d5b72b0ea7abe7ed80" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 39509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:50.324623 26008 consensus_queue.cc:1035] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b098a6fe10494d3591299acd185a08d7" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:50.332751 26018 mvcc.cc:205] Tried to move back new op lower bound from 7316293838119079936 to 7316293838048288768. Current Snapshot: MvccSnapshot[applied={T|T < 7316293838119079936}]
I20260808 15:55:50.333338 26017 mvcc.cc:205] Tried to move back new op lower bound from 7316293838119079936 to 7316293838048288768. Current Snapshot: MvccSnapshot[applied={T|T < 7316293838119079936}]
I20260808 15:55:50.341394 25361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:50.341538 25361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:50.343971 25361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c9ffb79d6e3455f84f19f8b81ae33d5: Sending DeleteTablet for 3 replicas of tablet 862412c9f0b645a99f0b8399e898e48e
I20260808 15:55:50.344791 25516 tablet_service.cc:1514] Processing DeleteTablet for tablet 862412c9f0b645a99f0b8399e898e48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:50 UTC) from {username='slave'} at 127.0.0.1:55642
I20260808 15:55:50.344820 25888 tablet_service.cc:1514] Processing DeleteTablet for tablet 862412c9f0b645a99f0b8399e898e48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:50 UTC) from {username='slave'} at 127.0.0.1:48820
I20260808 15:55:50.345005 25699 tablet_service.cc:1514] Processing DeleteTablet for tablet 862412c9f0b645a99f0b8399e898e48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:50 UTC) from {username='slave'} at 127.0.0.1:51254
I20260808 15:55:50.345353 26031 tablet_replica.cc:330] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: stopping tablet replica
I20260808 15:55:50.345623 26032 tablet_replica.cc:330] T 862412c9f0b645a99f0b8399e898e48e P b098a6fe10494d3591299acd185a08d7: stopping tablet replica
I20260808 15:55:50.345681 26031 raft_consensus.cc:2229] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:50.345712 26033 tablet_replica.cc:330] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: stopping tablet replica
I20260808 15:55:50.346025 26033 raft_consensus.cc:2229] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:55:50.346071 26031 raft_consensus.cc:2258] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:50.346134 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 25431
I20260808 15:55:50.349162 26033 raft_consensus.cc:2258] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:50.350205 26031 ts_tablet_manager.cc:1905] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:50.353598 26031 ts_tablet_manager.cc:1918] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:50.353700 26031 log.cc:1196] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/862412c9f0b645a99f0b8399e898e48e
I20260808 15:55:50.353971 26031 ts_tablet_manager.cc:1939] T 862412c9f0b645a99f0b8399e898e48e P 6160ba053ad546d5b72b0ea7abe7ed80: Deleting consensus metadata
I20260808 15:55:50.355209 25341 catalog_manager.cc:4841] TS 6160ba053ad546d5b72b0ea7abe7ed80 (127.30.76.67:39509): tablet 862412c9f0b645a99f0b8399e898e48e (table test-table [id=7c9ea865c6cd4ef2bf95359f3a9812e8]) successfully deleted
I20260808 15:55:50.358381 26033 ts_tablet_manager.cc:1905] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:50.361932 26033 ts_tablet_manager.cc:1918] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:50.362040 26033 log.cc:1196] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/862412c9f0b645a99f0b8399e898e48e
I20260808 15:55:50.362360 26033 ts_tablet_manager.cc:1939] T 862412c9f0b645a99f0b8399e898e48e P 2c977154e8fd435aa1486eb1a5f51f63: Deleting consensus metadata
W20260808 15:55:50.363469 25339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:39953 (error 108)
I20260808 15:55:50.364068 25344 catalog_manager.cc:4841] TS 2c977154e8fd435aa1486eb1a5f51f63 (127.30.76.66:38107): tablet 862412c9f0b645a99f0b8399e898e48e (table test-table [id=7c9ea865c6cd4ef2bf95359f3a9812e8]) successfully deleted
I20260808 15:55:50.364544 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 25609
I20260808 15:55:50.364715 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:50.364430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:39953 (local address 127.0.0.1:55652)
0808 15:55:50.364545 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:50.364613 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:39953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 15:55:50.368811 25339 catalog_manager.cc:4584] TS b098a6fe10494d3591299acd185a08d7 (127.30.76.65:39953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 862412c9f0b645a99f0b8399e898e48e: Network error: Client connection negotiation failed: client connection to 127.30.76.65:39953: connect: Connection refused (error 111)
I20260808 15:55:50.376135 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 25795
I20260808 15:55:50.384268 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 25316
2026-08-08T15:55:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1060 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:50 dist-test-slave-sb2z krb5kdc[26040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:50 dist-test-slave-sb2z krb5kdc[26040](info): set up 2 sockets
Aug 08 15:55:50 dist-test-slave-sb2z krb5kdc[26040](info): commencing operation
krb5kdc: starting...
W20260808 15:55:51.616465 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:55:51 dist-test-slave-sb2z krb5kdc[26040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:55:51.742832 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:46223
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:46223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:51.856635 26059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:51.856912 26059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:51.856972 26059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:51.860910 26059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:51.860992 26059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:51.861012 26059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:51.861028 26059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:51.861043 26059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:51.866058 26059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:46223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:46223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:51.867004 26059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:51.867970 26059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:51.876084 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:51.876191 26059 server_base.cc:1034] running on GCE node
W20260808 15:55:51.876150 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:51.876122 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:51.876916 26059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:51.877965 26059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:51.879122 26059 hybrid_clock.cc:648] HybridClock initialized: now 1786204551879098 us; error 39 us; skew 500 ppm
Aug 08 15:55:51 dist-test-slave-sb2z krb5kdc[26040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204551, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:51.886977 26059 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:55:51.887912 26059 webserver.cc:460] Webserver started at http://127.30.76.126:38851/ using document root <none> and password file <none>
I20260808 15:55:51.888466 26059 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:51.888528 26059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:51.888806 26059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:51.890797 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "0c57321e023e4dfe8970d42f248ef47b"
format_stamp: "Formatted at 2026-08-08 15:55:51 on dist-test-slave-sb2z"
server_key: "df5d2ec71807e60516916df5d51faff5"
server_key_iv: "d8e9d4cd198934fd46ad35575f8f78fd"
server_key_version: "encryptionkey@0"
I20260808 15:55:51.891384 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "0c57321e023e4dfe8970d42f248ef47b"
format_stamp: "Formatted at 2026-08-08 15:55:51 on dist-test-slave-sb2z"
server_key: "df5d2ec71807e60516916df5d51faff5"
server_key_iv: "d8e9d4cd198934fd46ad35575f8f78fd"
server_key_version: "encryptionkey@0"
I20260808 15:55:51.895087 26059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:55:51.897428 26077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:51.898582 26059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:55:51.898739 26059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "0c57321e023e4dfe8970d42f248ef47b"
format_stamp: "Formatted at 2026-08-08 15:55:51 on dist-test-slave-sb2z"
server_key: "df5d2ec71807e60516916df5d51faff5"
server_key_iv: "d8e9d4cd198934fd46ad35575f8f78fd"
server_key_version: "encryptionkey@0"
I20260808 15:55:51.898867 26059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:51.918352 26059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:51.921314 26059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:51.921523 26059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:51.928435 26059 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:46223
I20260808 15:55:51.928449 26148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:46223 every 8 connection(s)
I20260808 15:55:51.929354 26059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:51.932107 26149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:51.937351 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b: Bootstrap starting.
I20260808 15:55:51.939704 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 26059
I20260808 15:55:51.939867 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:51.939874 26149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:51.940235 31025 external_mini_cluster.cc:1428] Setting key f57704ed322dcc2f3cbb47dfff3585df
I20260808 15:55:51.940776 26149 log.cc:824] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:51.942770 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b: No bootstrap required, opened a new log
I20260808 15:55:51.946198 26149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c57321e023e4dfe8970d42f248ef47b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46223 } }
I20260808 15:55:51.946437 26149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:51.946527 26149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c57321e023e4dfe8970d42f248ef47b, State: Initialized, Role: FOLLOWER
I20260808 15:55:51.946998 26149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c57321e023e4dfe8970d42f248ef47b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46223 } }
I20260808 15:55:51.947155 26149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:51.947252 26149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:51.947350 26149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:51.948295 26149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c57321e023e4dfe8970d42f248ef47b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46223 } }
I20260808 15:55:51.948668 26149 leader_election.cc:302] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c57321e023e4dfe8970d42f248ef47b; no voters:
I20260808 15:55:51.949039 26149 leader_election.cc:288] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:51.949112 26154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:51.949359 26154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [term 1 LEADER]: Becoming Leader. State: Replica: 0c57321e023e4dfe8970d42f248ef47b, State: Running, Role: LEADER
I20260808 15:55:51.949710 26154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c57321e023e4dfe8970d42f248ef47b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46223 } }
Aug 08 15:55:51 dist-test-slave-sb2z krb5kdc[26040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:51.950294 26149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:51.950865 26154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c57321e023e4dfe8970d42f248ef47b. Latest consensus state: current_term: 1 leader_uuid: "0c57321e023e4dfe8970d42f248ef47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c57321e023e4dfe8970d42f248ef47b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46223 } } }
I20260808 15:55:51.950982 26154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:51.951804 26165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:51.952919 26155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c57321e023e4dfe8970d42f248ef47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c57321e023e4dfe8970d42f248ef47b" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 46223 } } }
I20260808 15:55:51.953161 26155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:51.955343 26165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:51.956182 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:51.941398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.30.76.126:46223)
0808 15:55:51.941800 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:51.941822 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:51.941857 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:55:51.943657 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:51.943676 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:51.943975 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:51.944313 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:51.945154 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:51.946082 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:51.947122 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:51.947813 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:51.947994 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:51.950858 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:51.950863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:51.950877 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:51.950903 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:51.953086 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:51.953610 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:51.953616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:51.953622 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:51.953775 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:51.954080 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:51.954085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:51.954087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:51.954568 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:51.954776 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:51.955094 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 15:55:51.955400 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":137,"threads_started":1}
I20260808 15:55:51.961191 26165 catalog_manager.cc:1347] Generated new cluster ID: 0edfc5714ccb4f02883fd69c4cc78709
I20260808 15:55:51.961254 26165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:51.970176 26165 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:51.971177 26165 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:51.978858 26165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b: Generated new TSK 0
I20260808 15:55:51.979475 26165 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
Entry for principal kudu/127.30.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab.
I20260808 15:55:52.014863 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46223
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:52.142565 26186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:52.142813 26186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:52.142856 26186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:52.146800 26186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:52.146858 26186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:52.146914 26186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:52.152374 26186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.65.keytab
--principal=kudu/127.30.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:52.153474 26186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:52.154489 26186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:52.164362 26194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:52.164422 26193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:52.164362 26200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:52.164572 26186 server_base.cc:1034] running on GCE node
I20260808 15:55:52.164996 26186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:52.165642 26186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:52.166808 26186 hybrid_clock.cc:648] HybridClock initialized: now 1786204552166779 us; error 42 us; skew 500 ppm
Aug 08 15:55:52 dist-test-slave-sb2z krb5kdc[26040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204552, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:52.169745 26186 init.cc:377] Logged in from keytab as kudu/127.30.76.65@KRBTEST.COM (short username kudu)
I20260808 15:55:52.170578 26186 webserver.cc:460] Webserver started at http://127.30.76.65:36787/ using document root <none> and password file <none>
I20260808 15:55:52.171164 26186 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:52.171264 26186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:52.171506 26186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:52.173552 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "1d48c9455ebd4f6a8a156d574bbedd38"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "537caff4b5a598e3e7765dd493bba8f1"
server_key_iv: "ed5399b3d061713c8efd22eb35a596fe"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.174134 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "1d48c9455ebd4f6a8a156d574bbedd38"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "537caff4b5a598e3e7765dd493bba8f1"
server_key_iv: "ed5399b3d061713c8efd22eb35a596fe"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.178013 26186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 15:55:52.180464 26210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.181689 26186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:52.181816 26186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "1d48c9455ebd4f6a8a156d574bbedd38"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "537caff4b5a598e3e7765dd493bba8f1"
server_key_iv: "ed5399b3d061713c8efd22eb35a596fe"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.181922 26186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:52.194530 26186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:52.197873 26186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:52.198091 26186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:52.198704 26186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:52.199553 26186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:52.199600 26186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.199684 26186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:52.199718 26186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.208992 26186 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:44333
I20260808 15:55:52.209028 26373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:44333 every 8 connection(s)
I20260808 15:55:52.209913 26186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:52.213778 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 26186
I20260808 15:55:52.213943 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:55:52.214278 31025 external_mini_cluster.cc:1428] Setting key 795685de9f8fb2c9cd5c77feb99182db
Aug 08 15:55:52 dist-test-slave-sb2z krb5kdc[26040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204552, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.65@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.66@KRBTEST.COM; defaulting to no policy
I20260808 15:55:52.226056 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.211938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:53665 (local address 127.30.76.126:46223)
0808 15:55:52.212105 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.212110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.213029 (+ 919us) server_negotiation.cc:407] Connection header received
0808 15:55:52.214714 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.214718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.214779 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.214878 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:52.216669 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.217187 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.218696 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.218951 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.219080 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.221957 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:52.221966 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:52.221968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:52.221994 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:52.223793 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.224335 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.224339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.224340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.224431 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.224804 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.224809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.224811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.225016 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:52.225144 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.225721 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.225871 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:55:52.227064 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.212381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:46223 (local address 127.30.76.65:53665)
0808 15:55:52.212856 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.212899 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.214448 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.215244 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.215250 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.215820 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:52.216501 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.216525 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.217331 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.217337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.218539 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.218550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.219123 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.219128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.219367 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.220068 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:52.220084 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:52.221767 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:52.223921 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:52.223928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:52.223944 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:52.224217 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:52.224534 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:52.224538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:52.224540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:52.224697 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:52.225134 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:52.225141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:52.225457 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.225677 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.226023 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.66@KRBTEST.COM" created.
I20260808 15:55:52.228513 26375 heartbeater.cc:347] Connected to a master server at 127.30.76.126:46223
I20260808 15:55:52.228824 26375 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:52.229454 26375 heartbeater.cc:511] Master 127.30.76.126:46223 requested a full tablet report, sending...
I20260808 15:55:52.231382 26102 ts_manager.cc:194] Registered new tserver with Master: 1d48c9455ebd4f6a8a156d574bbedd38 (127.30.76.65:44333)
I20260808 15:55:52.232724 26102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.65@KRBTEST.COM'} at 127.30.76.65:53665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
Entry for principal kudu/127.30.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab.
I20260808 15:55:52.246019 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46223
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:52.357060 26381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:52.357349 26381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:52.357429 26381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:52.361682 26381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:52.361768 26381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:52.361852 26381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:52.366933 26381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.66.keytab
--principal=kudu/127.30.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:52.368005 26381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:52.368991 26381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:52.378568 26391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:52.378568 26388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:52.378638 26393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:52.378973 26381 server_base.cc:1034] running on GCE node
I20260808 15:55:52.379362 26381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:52.380036 26381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:52.381251 26381 hybrid_clock.cc:648] HybridClock initialized: now 1786204552381239 us; error 26 us; skew 500 ppm
Aug 08 15:55:52 dist-test-slave-sb2z krb5kdc[26040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204552, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:52.384115 26381 init.cc:377] Logged in from keytab as kudu/127.30.76.66@KRBTEST.COM (short username kudu)
I20260808 15:55:52.385004 26381 webserver.cc:460] Webserver started at http://127.30.76.66:41107/ using document root <none> and password file <none>
I20260808 15:55:52.385565 26381 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:52.385666 26381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:52.385947 26381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:52.388094 26381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "de33c42662fd408983a177bb24979824"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "eda87724ab51ad50cedfe41a0466be96"
server_key_iv: "e83a5aaadd1c01e87ee6c5e9e3aa38f5"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.388669 26381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "de33c42662fd408983a177bb24979824"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "eda87724ab51ad50cedfe41a0466be96"
server_key_iv: "e83a5aaadd1c01e87ee6c5e9e3aa38f5"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.392428 26381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:55:52.394807 26400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.396106 26381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:55:52.396256 26381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "de33c42662fd408983a177bb24979824"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "eda87724ab51ad50cedfe41a0466be96"
server_key_iv: "e83a5aaadd1c01e87ee6c5e9e3aa38f5"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.396371 26381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:52.424693 26381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:52.428334 26381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:52.428591 26381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:52.429247 26381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:52.430158 26381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:52.430241 26381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.430315 26381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:52.430351 26381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.439741 26381 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:37843
I20260808 15:55:52.439762 26560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:37843 every 8 connection(s)
I20260808 15:55:52.440692 26381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:52.442870 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 26381
I20260808 15:55:52.443033 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:55:52.443351 31025 external_mini_cluster.cc:1428] Setting key c7825d0e817b877ae4f5ce302e4c94bc
Aug 08 15:55:52 dist-test-slave-sb2z krb5kdc[26040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204552, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.66@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.67@KRBTEST.COM; defaulting to no policy
I20260808 15:55:52.457461 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.442712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:38825 (local address 127.30.76.126:46223)
0808 15:55:52.442876 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.442882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.444169 (+ 1287us) server_negotiation.cc:407] Connection header received
0808 15:55:52.445788 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.445791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.445837 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.445912 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:52.448020 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.448543 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.449724 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.449944 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.450061 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.452993 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:52.452999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:52.453005 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:52.453041 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:52.454957 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.455553 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.455559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.455561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.455664 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.456175 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.456179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.456183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.456373 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:52.456513 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.457135 (+ 622us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.457265 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:55:52.458333 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.443213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:46223 (local address 127.30.76.66:38825)
0808 15:55:52.443989 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.444047 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.445509 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.446139 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.446148 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.446837 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:52.447852 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.447872 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.448696 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.448699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.449579 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.449590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.450110 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.450115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.450281 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.450943 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:52.450959 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:52.452817 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:52.455099 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:52.455104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:52.455117 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:52.455381 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:52.455806 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:52.455809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:52.455812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:52.456054 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:52.456521 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:52.456530 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:52.456799 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.457073 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.457422 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":147,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.67@KRBTEST.COM" created.
I20260808 15:55:52.459714 26561 heartbeater.cc:347] Connected to a master server at 127.30.76.126:46223
I20260808 15:55:52.460028 26561 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:52.460544 26561 heartbeater.cc:511] Master 127.30.76.126:46223 requested a full tablet report, sending...
I20260808 15:55:52.461796 26102 ts_manager.cc:194] Registered new tserver with Master: de33c42662fd408983a177bb24979824 (127.30.76.66:37843)
I20260808 15:55:52.462415 26102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.66@KRBTEST.COM'} at 127.30.76.66:38825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
Entry for principal kudu/127.30.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab.
I20260808 15:55:52.476495 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46223
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:52.592444 26571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:52.592674 26571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:52.592749 26571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:52.596740 26571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:52.596796 26571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:52.596894 26571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:52.602308 26571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.67.keytab
--principal=kudu/127.30.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:46223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:52.603384 26571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:52.604440 26571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:52.613946 26580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:52.613946 26579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:52.613938 26582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:52.614092 26571 server_base.cc:1034] running on GCE node
I20260808 15:55:52.614708 26571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:52.615302 26571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:52.616478 26571 hybrid_clock.cc:648] HybridClock initialized: now 1786204552616439 us; error 47 us; skew 500 ppm
Aug 08 15:55:52 dist-test-slave-sb2z krb5kdc[26040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204552, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:52.619234 26571 init.cc:377] Logged in from keytab as kudu/127.30.76.67@KRBTEST.COM (short username kudu)
I20260808 15:55:52.620167 26571 webserver.cc:460] Webserver started at http://127.30.76.67:38053/ using document root <none> and password file <none>
I20260808 15:55:52.620746 26571 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:52.620810 26571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:52.621121 26571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:52.623132 26571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "9803874c138c46a2a44dd6a9a7389f67"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "cdb98f6b400546d940e1706ef0df1207"
server_key_iv: "428397ee47180c2e3bd579009e92ec68"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.623735 26571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "9803874c138c46a2a44dd6a9a7389f67"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "cdb98f6b400546d940e1706ef0df1207"
server_key_iv: "428397ee47180c2e3bd579009e92ec68"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.627357 26571 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:55:52.629739 26595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.630914 26571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:52.631071 26571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "9803874c138c46a2a44dd6a9a7389f67"
format_stamp: "Formatted at 2026-08-08 15:55:52 on dist-test-slave-sb2z"
server_key: "cdb98f6b400546d940e1706ef0df1207"
server_key_iv: "428397ee47180c2e3bd579009e92ec68"
server_key_version: "encryptionkey@0"
I20260808 15:55:52.631176 26571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:52.652565 26571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:52.655851 26571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:52.656013 26571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:52.656656 26571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:52.657485 26571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:52.657534 26571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.657620 26571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:52.657657 26571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:52.666708 26571 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:35587
I20260808 15:55:52.666744 26756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:35587 every 8 connection(s)
I20260808 15:55:52.667616 26571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:52.674067 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 26571
I20260808 15:55:52.674232 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:55:52.674537 31025 external_mini_cluster.cc:1428] Setting key e793a5416a2f6cf36acb5a44daf5382d
Aug 08 15:55:52 dist-test-slave-sb2z krb5kdc[26040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204552, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.67@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:55:52.681425 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.669455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:50909 (local address 127.30.76.126:46223)
0808 15:55:52.669602 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.669605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.670416 (+ 811us) server_negotiation.cc:407] Connection header received
0808 15:55:52.671751 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.671755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.671813 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.671933 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:52.673083 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.673629 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.675095 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.675420 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.675583 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.678166 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:52.678169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:52.678171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:52.678195 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:52.679450 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.680022 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.680026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.680028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.680129 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.680459 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.680462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.680463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.680634 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:52.680710 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.681205 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.681302 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:55:52.682133 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.669800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:46223 (local address 127.30.76.67:50909)
0808 15:55:52.670267 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.670309 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.671478 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.672105 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.672112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.672445 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:52.672923 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.672936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.673859 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.673862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.674961 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.674973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.675648 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.675653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.675801 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.676481 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:55:52.676497 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:52.678012 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:52.679629 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:52.679634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:52.679645 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:52.679886 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:52.680230 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:55:52.680232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:55:52.680233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:55:52.680359 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:55:52.680728 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:52.680732 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:52.680956 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.681107 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.681337 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":134,"threads_started":1}
I20260808 15:55:52.683291 26757 heartbeater.cc:347] Connected to a master server at 127.30.76.126:46223
I20260808 15:55:52.683568 26757 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:52.684042 26757 heartbeater.cc:511] Master 127.30.76.126:46223 requested a full tablet report, sending...
I20260808 15:55:52.685101 26102 ts_manager.cc:194] Registered new tserver with Master: 9803874c138c46a2a44dd6a9a7389f67 (127.30.76.67:35587)
I20260808 15:55:52.685729 26102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.76.67@KRBTEST.COM'} at 127.30.76.67:50909
I20260808 15:55:52.688611 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:52.697654 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.690549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.30.76.126:46223)
0808 15:55:52.690696 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.690700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.690902 (+ 202us) server_negotiation.cc:407] Connection header received
0808 15:55:52.690998 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.691000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.691037 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.691102 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:52.691871 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.692360 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.693201 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.693397 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.693470 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.694714 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:52.694717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:52.694719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:52.694740 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:52.696025 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.696442 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.696446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.696447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.696545 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:52.696826 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:52.696829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:52.696830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:52.697036 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:52.697153 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.697349 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.697439 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:55:52.700634 26102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:52.704099 26102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:52.718895 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.713485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.30.76.67:35587)
0808 15:55:52.714310 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.714323 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.714346 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:55:52.714425 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.714429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.714610 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.714801 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.715011 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.715812 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.717613 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.718162 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.718288 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.718436 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.718632 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.718725 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":158,"threads_started":1}
I20260808 15:55:52.718930 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.712674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:37843 (local address 127.0.0.1:50978)
0808 15:55:52.713490 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.713538 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.713716 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.714318 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.714324 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.714348 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.714493 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.714504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.716171 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.716174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.717466 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.717478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.718488 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.718492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.718576 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.718597 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.718665 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.718739 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":47,"thread_start_us":61,"threads_started":1}
I20260808 15:55:52.718912 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.713054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:35587 (local address 127.0.0.1:36644)
0808 15:55:52.713974 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.713987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.714057 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.714787 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.714790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.714808 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.714896 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.714902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.715942 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.715948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.717470 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.717478 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.718300 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.718304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.718406 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.718444 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.718583 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.718715 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":112,"thread_start_us":95,"threads_started":1}
I20260808 15:55:52.719274 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.713426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41294 (local address 127.30.76.65:44333)
0808 15:55:52.714088 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.714106 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.714129 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:55:52.714217 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.714225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.714397 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.714556 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.715041 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.716196 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.717523 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.718345 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.718465 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.718600 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.719010 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.719114 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":117,"threads_started":1}
I20260808 15:55:52.719681 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.712760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44333 (local address 127.0.0.1:41294)
0808 15:55:52.713542 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.713560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.713700 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.714542 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.714545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.714564 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.714691 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.714701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.716334 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.716340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.717385 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.717395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.718663 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.718667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.718745 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.718765 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.718836 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.718889 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":4,"thread_start_us":107,"threads_started":1}
I20260808 15:55:52.719755 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.712786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.30.76.66:37843)
0808 15:55:52.713218 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.713231 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.713633 (+ 402us) server_negotiation.cc:407] Connection header received
0808 15:55:52.713913 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.713921 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.714136 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.714648 (+ 512us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.714801 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.716022 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.717575 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.718345 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.719280 (+ 935us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.719411 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.719459 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.719601 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":131,"threads_started":1}
I20260808 15:55:52.722218 26664 tablet_service.cc:1467] Processing CreateTablet for tablet acec3cd7173844d8a4e453f8173e0406 (DEFAULT_TABLE table=test-table [id=e783b256e3be411787ead2f15702510e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:52.722519 26470 tablet_service.cc:1467] Processing CreateTablet for tablet acec3cd7173844d8a4e453f8173e0406 (DEFAULT_TABLE table=test-table [id=e783b256e3be411787ead2f15702510e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:52.723250 26664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acec3cd7173844d8a4e453f8173e0406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:52.723027 26282 tablet_service.cc:1467] Processing CreateTablet for tablet acec3cd7173844d8a4e453f8173e0406 (DEFAULT_TABLE table=test-table [id=e783b256e3be411787ead2f15702510e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:52.723551 26470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acec3cd7173844d8a4e453f8173e0406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:52.724030 26282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acec3cd7173844d8a4e453f8173e0406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:52.729439 26778 tablet_bootstrap.cc:492] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Bootstrap starting.
I20260808 15:55:52.730888 26780 tablet_bootstrap.cc:492] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: Bootstrap starting.
I20260808 15:55:52.731417 26779 tablet_bootstrap.cc:492] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Bootstrap starting.
I20260808 15:55:52.732019 26778 tablet_bootstrap.cc:654] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:52.733088 26780 tablet_bootstrap.cc:654] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:52.733354 26778 log.cc:824] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:52.733639 26779 tablet_bootstrap.cc:654] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:52.734059 26780 log.cc:824] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:52.734831 26779 log.cc:824] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:52.735613 26778 tablet_bootstrap.cc:492] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: No bootstrap required, opened a new log
I20260808 15:55:52.735848 26778 ts_tablet_manager.cc:1397] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:55:52.736100 26780 tablet_bootstrap.cc:492] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: No bootstrap required, opened a new log
I20260808 15:55:52.736228 26780 ts_tablet_manager.cc:1397] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260808 15:55:52.736837 26779 tablet_bootstrap.cc:492] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: No bootstrap required, opened a new log
I20260808 15:55:52.736995 26779 ts_tablet_manager.cc:1397] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 15:55:52.739583 26780 raft_consensus.cc:348] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.739847 26780 raft_consensus.cc:374] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:52.739933 26780 raft_consensus.cc:729] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d48c9455ebd4f6a8a156d574bbedd38, State: Initialized, Role: FOLLOWER
I20260808 15:55:52.740136 26778 raft_consensus.cc:348] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.740402 26778 raft_consensus.cc:374] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:52.740404 26780 consensus_queue.cc:260] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.740502 26778 raft_consensus.cc:729] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9803874c138c46a2a44dd6a9a7389f67, State: Initialized, Role: FOLLOWER
I20260808 15:55:52.740721 26779 raft_consensus.cc:348] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.740995 26779 raft_consensus.cc:374] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:52.740916 26778 consensus_queue.cc:260] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.741092 26779 raft_consensus.cc:729] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de33c42662fd408983a177bb24979824, State: Initialized, Role: FOLLOWER
I20260808 15:55:52.741470 26780 ts_tablet_manager.cc:1428] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260808 15:55:52.741600 26757 heartbeater.cc:503] Master 127.30.76.126:46223 was elected leader, sending a full tablet report...
I20260808 15:55:52.741544 26779 consensus_queue.cc:260] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.741938 26778 ts_tablet_manager.cc:1428] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 15:55:52.741984 26375 heartbeater.cc:503] Master 127.30.76.126:46223 was elected leader, sending a full tablet report...
I20260808 15:55:52.742429 26779 ts_tablet_manager.cc:1428] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 15:55:52.742731 26561 heartbeater.cc:503] Master 127.30.76.126:46223 was elected leader, sending a full tablet report...
I20260808 15:55:52.850371 26786 raft_consensus.cc:482] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:52.850688 26786 raft_consensus.cc:504] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.852095 26786 leader_election.cc:288] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de33c42662fd408983a177bb24979824 (127.30.76.66:37843), 1d48c9455ebd4f6a8a156d574bbedd38 (127.30.76.65:44333)
I20260808 15:55:52.856079 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.852312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:37843 (local address 127.30.76.67:37125)
0808 15:55:52.852496 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.852515 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.852613 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.852966 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.852969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.852994 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.853137 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.853144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.853998 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.854001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.854832 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.854842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.855580 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.855586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.855744 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.855768 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.855832 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.855884 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":23}
I20260808 15:55:52.856096 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.852437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:37125 (local address 127.30.76.66:37843)
0808 15:55:52.852567 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.852572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.852591 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:55:52.852762 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.852765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.852825 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.852921 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.853269 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.853879 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.854965 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.855427 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.855576 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.855647 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.855877 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.855957 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:55:52.856146 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.852363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44333 (local address 127.30.76.67:55729)
0808 15:55:52.852850 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.852866 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.852967 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.853266 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.853268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.853287 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.853395 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.853402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.854159 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.854162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.854966 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.854975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.855690 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.855694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.855793 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.855837 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.855919 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.855983 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":2,"thread_start_us":127,"threads_started":1}
I20260808 15:55:52.856159 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.852533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:55729 (local address 127.30.76.65:44333)
0808 15:55:52.852659 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.852664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.852960 (+ 296us) server_negotiation.cc:407] Connection header received
0808 15:55:52.853073 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.853076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.853143 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.853232 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.853510 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.854041 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.855092 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.855512 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.855645 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.855697 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.855959 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.856030 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:55:52.856909 26308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acec3cd7173844d8a4e453f8173e0406" candidate_uuid: "9803874c138c46a2a44dd6a9a7389f67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" is_pre_election: true
I20260808 15:55:52.856943 26499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acec3cd7173844d8a4e453f8173e0406" candidate_uuid: "9803874c138c46a2a44dd6a9a7389f67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de33c42662fd408983a177bb24979824" is_pre_election: true
I20260808 15:55:52.857215 26308 raft_consensus.cc:2454] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9803874c138c46a2a44dd6a9a7389f67 in term 0.
I20260808 15:55:52.857215 26499 raft_consensus.cc:2454] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9803874c138c46a2a44dd6a9a7389f67 in term 0.
I20260808 15:55:52.857370 26787 raft_consensus.cc:482] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:52.857510 26787 raft_consensus.cc:504] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.857728 26602 leader_election.cc:302] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d48c9455ebd4f6a8a156d574bbedd38, 9803874c138c46a2a44dd6a9a7389f67; no voters:
I20260808 15:55:52.858040 26786 raft_consensus.cc:2781] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:52.858126 26786 raft_consensus.cc:482] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:55:52.858188 26786 raft_consensus.cc:3037] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:52.858628 26787 leader_election.cc:288] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d48c9455ebd4f6a8a156d574bbedd38 (127.30.76.65:44333), 9803874c138c46a2a44dd6a9a7389f67 (127.30.76.67:35587)
I20260808 15:55:52.859648 26786 raft_consensus.cc:504] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.860059 26786 leader_election.cc:288] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [CANDIDATE]: Term 1 election: Requested vote from peers de33c42662fd408983a177bb24979824 (127.30.76.66:37843), 1d48c9455ebd4f6a8a156d574bbedd38 (127.30.76.65:44333)
I20260808 15:55:52.860486 26499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acec3cd7173844d8a4e453f8173e0406" candidate_uuid: "9803874c138c46a2a44dd6a9a7389f67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de33c42662fd408983a177bb24979824"
I20260808 15:55:52.860486 26308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acec3cd7173844d8a4e453f8173e0406" candidate_uuid: "9803874c138c46a2a44dd6a9a7389f67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d48c9455ebd4f6a8a156d574bbedd38"
I20260808 15:55:52.860603 26499 raft_consensus.cc:3037] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:52.860603 26308 raft_consensus.cc:3037] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:52.861881 26499 raft_consensus.cc:2454] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9803874c138c46a2a44dd6a9a7389f67 in term 1.
I20260808 15:55:52.861881 26308 raft_consensus.cc:2454] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9803874c138c46a2a44dd6a9a7389f67 in term 1.
I20260808 15:55:52.862263 26602 leader_election.cc:302] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d48c9455ebd4f6a8a156d574bbedd38, 9803874c138c46a2a44dd6a9a7389f67; no voters:
I20260808 15:55:52.862398 26786 raft_consensus.cc:2781] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:52.862644 26786 raft_consensus.cc:686] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 1 LEADER]: Becoming Leader. State: Replica: 9803874c138c46a2a44dd6a9a7389f67, State: Running, Role: LEADER
I20260808 15:55:52.862962 26786 consensus_queue.cc:237] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } }
I20260808 15:55:52.863266 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.858856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:48993 (local address 127.30.76.65:44333)
0808 15:55:52.858966 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.858972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.859204 (+ 232us) server_negotiation.cc:407] Connection header received
0808 15:55:52.859309 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.859311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.859374 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.859445 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.859967 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.860655 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.862138 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.862681 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.862778 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.862819 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.863096 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.863164 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 15:55:52.863315 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.858817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44333 (local address 127.30.76.66:48993)
0808 15:55:52.859112 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.859126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.859220 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.859669 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.859672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.859688 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.859837 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.859843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.860773 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.860776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.861991 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.862001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.862873 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.862876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.862978 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.863000 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.863074 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.863181 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"spinlock_wait_cycles":27904}
I20260808 15:55:52.863757 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.859449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:35233 (local address 127.30.76.67:35587)
0808 15:55:52.859611 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.859615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.859731 (+ 116us) server_negotiation.cc:407] Connection header received
0808 15:55:52.859906 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.859912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.859959 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.860024 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:52.860380 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.861078 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.862516 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.863071 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.863149 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.863190 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.863562 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.863642 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:55:52.863802 26308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acec3cd7173844d8a4e453f8173e0406" candidate_uuid: "de33c42662fd408983a177bb24979824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" is_pre_election: true
I20260808 15:55:52.863952 26308 raft_consensus.cc:2379] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de33c42662fd408983a177bb24979824 in current term 1: Already voted for candidate 9803874c138c46a2a44dd6a9a7389f67 in this term.
I20260808 15:55:52.864094 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.858839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:35587 (local address 127.30.76.66:35233)
0808 15:55:52.859616 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.859632 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:52.859814 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:52.860112 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:52.860115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:52.860131 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:52.860260 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.860266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.861225 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.861228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.862368 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:52.862377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.863279 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.863282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:52.863385 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.863406 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:55:52.863926 (+ 520us) client_negotiation.cc:240] Negotiation successful
0808 15:55:52.863998 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"spinlock_wait_cycles":896,"thread_start_us":117,"threads_started":1}
I20260808 15:55:52.864603 26692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acec3cd7173844d8a4e453f8173e0406" candidate_uuid: "de33c42662fd408983a177bb24979824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9803874c138c46a2a44dd6a9a7389f67" is_pre_election: true
I20260808 15:55:52.865213 26403 leader_election.cc:302] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de33c42662fd408983a177bb24979824; no voters: 1d48c9455ebd4f6a8a156d574bbedd38, 9803874c138c46a2a44dd6a9a7389f67
I20260808 15:55:52.865646 26787 raft_consensus.cc:2726] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 15:55:52.866343 26099 catalog_manager.cc:5495] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9803874c138c46a2a44dd6a9a7389f67 (127.30.76.67). New cstate: current_term: 1 leader_uuid: "9803874c138c46a2a44dd6a9a7389f67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9803874c138c46a2a44dd6a9a7389f67" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 35587 } health_report { overall_health: HEALTHY } } }
I20260808 15:55:52.912542 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.908899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.30.76.67:35587)
0808 15:55:52.909026 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:52.909031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:52.909046 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:55:52.909177 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:52.909181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:52.909241 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:52.909358 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:52.909625 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.910115 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.910996 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:52.911231 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:52.911352 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:52.911728 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:52.912112 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:52.912263 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:52.912330 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 15:55:52.912381 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 15:55:52.918205 26308 raft_consensus.cc:1261] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 1 FOLLOWER]: Refusing update from remote peer 9803874c138c46a2a44dd6a9a7389f67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:52.918249 26499 raft_consensus.cc:1261] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 1 FOLLOWER]: Refusing update from remote peer 9803874c138c46a2a44dd6a9a7389f67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 15:55:52.918591 26758 tablet.cc:2369] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:52.918923 26791 consensus_queue.cc:1035] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de33c42662fd408983a177bb24979824" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 37843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:52.919090 26786 consensus_queue.cc:1035] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d48c9455ebd4f6a8a156d574bbedd38" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 44333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:52.929798 26797 mvcc.cc:205] Tried to move back new op lower bound from 7316293848745684992 to 7316293848529858560. Current Snapshot: MvccSnapshot[applied={T|T < 7316293848745684992}]
I20260808 15:55:52.937080 26100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:52.937245 26100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:52.939728 26100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c57321e023e4dfe8970d42f248ef47b: Sending DeleteTablet for 3 replicas of tablet acec3cd7173844d8a4e453f8173e0406
I20260808 15:55:52.940486 26470 tablet_service.cc:1514] Processing DeleteTablet for tablet acec3cd7173844d8a4e453f8173e0406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:52 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:50978
I20260808 15:55:52.940490 26664 tablet_service.cc:1514] Processing DeleteTablet for tablet acec3cd7173844d8a4e453f8173e0406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:52 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:36644
I20260808 15:55:52.940642 26282 tablet_service.cc:1514] Processing DeleteTablet for tablet acec3cd7173844d8a4e453f8173e0406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:52 UTC) from {username='kudu', principal='kudu/127.30.76.126@KRBTEST.COM'} at 127.0.0.1:41294
I20260808 15:55:52.940950 26807 tablet_replica.cc:330] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: stopping tablet replica
I20260808 15:55:52.941237 26807 raft_consensus.cc:2229] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:52.941481 26812 tablet_replica.cc:330] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38: stopping tablet replica
I20260808 15:55:52.941497 26807 raft_consensus.cc:2258] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:52.941869 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 26186
I20260808 15:55:52.942272 26812 raft_consensus.cc:2229] T acec3cd7173844d8a4e453f8173e0406 P 1d48c9455ebd4f6a8a156d574bbedd38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:52.942832 26807 ts_tablet_manager.cc:1905] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:52.944774 26808 tablet_replica.cc:330] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: stopping tablet replica
I20260808 15:55:52.945220 26808 raft_consensus.cc:2229] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:55:52.945690 26807 ts_tablet_manager.cc:1918] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:52.945787 26807 log.cc:1196] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/acec3cd7173844d8a4e453f8173e0406
I20260808 15:55:52.945719 26808 raft_consensus.cc:2258] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:52.946413 26807 ts_tablet_manager.cc:1939] T acec3cd7173844d8a4e453f8173e0406 P de33c42662fd408983a177bb24979824: Deleting consensus metadata
I20260808 15:55:52.948328 26079 catalog_manager.cc:4841] TS de33c42662fd408983a177bb24979824 (127.30.76.66:37843): tablet acec3cd7173844d8a4e453f8173e0406 (table test-table [id=e783b256e3be411787ead2f15702510e]) successfully deleted
I20260808 15:55:52.948638 26808 ts_tablet_manager.cc:1905] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:52.952745 26808 ts_tablet_manager.cc:1918] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:52.952975 26808 log.cc:1196] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/acec3cd7173844d8a4e453f8173e0406
I20260808 15:55:52.953490 26808 ts_tablet_manager.cc:1939] T acec3cd7173844d8a4e453f8173e0406 P 9803874c138c46a2a44dd6a9a7389f67: Deleting consensus metadata
I20260808 15:55:52.956674 26083 catalog_manager.cc:4841] TS 9803874c138c46a2a44dd6a9a7389f67 (127.30.76.67:35587): tablet acec3cd7173844d8a4e453f8173e0406 (table test-table [id=e783b256e3be411787ead2f15702510e]) successfully deleted
W20260808 15:55:52.963718 26084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:44333 (error 108)
I20260808 15:55:52.964665 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 26381
I20260808 15:55:52.964883 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:52.964537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:44333 (local address 127.0.0.1:41310)
0808 15:55:52.964695 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:52.964781 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:44333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 15:55:52.969621 26084 catalog_manager.cc:4584] TS 1d48c9455ebd4f6a8a156d574bbedd38 (127.30.76.65:44333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet acec3cd7173844d8a4e453f8173e0406: Network error: Client connection negotiation failed: client connection to 127.30.76.65:44333: connect: Connection refused (error 111)
I20260808 15:55:52.976125 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 26571
I20260808 15:55:52.987301 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 26059
2026-08-08T15:55:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2604 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T15:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:53Z Disabled control of system clock
I20260808 15:55:53.021234 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41467
--webserver_interface=127.30.76.126
--webserver_port=0
--builtin_ntp_servers=127.30.76.84:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:41467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:55:53.131139 26819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:53.131387 26819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:53.131420 26819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:53.135984 26819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:53.136041 26819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:53.136056 26819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:53.136073 26819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:53.136088 26819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:53.140800 26819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:41467
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:41467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:53.141837 26819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:53.143168 26819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:53.151434 26828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:53.151443 26831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:53.151757 26819 server_base.cc:1034] running on GCE node
W20260808 15:55:53.151780 26829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:53.152449 26819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:53.153427 26819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:53.154598 26819 hybrid_clock.cc:648] HybridClock initialized: now 1786204553154572 us; error 38 us; skew 500 ppm
I20260808 15:55:53.156649 26819 webserver.cc:460] Webserver started at http://127.30.76.126:35103/ using document root <none> and password file <none>
I20260808 15:55:53.157227 26819 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:53.157289 26819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:53.157548 26819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:53.159622 26819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "029e46f63c4048a29a673198454fc882"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "e190e9ba1ec3cc0f2011c67c9c32c8ab"
server_key_iv: "195c31ed6cd53647a8e6ab7290704c38"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.160207 26819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "029e46f63c4048a29a673198454fc882"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "e190e9ba1ec3cc0f2011c67c9c32c8ab"
server_key_iv: "195c31ed6cd53647a8e6ab7290704c38"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.164019 26819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:55:53.166291 26839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.167443 26819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:53.167611 26819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "029e46f63c4048a29a673198454fc882"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "e190e9ba1ec3cc0f2011c67c9c32c8ab"
server_key_iv: "195c31ed6cd53647a8e6ab7290704c38"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.167727 26819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:53.183929 26819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:53.184567 26819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:53.184773 26819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:53.191828 26819 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:41467
I20260808 15:55:53.191869 26913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:41467 every 8 connection(s)
I20260808 15:55:53.192756 26819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:53.195855 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:53.197122 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 26819
I20260808 15:55:53.197278 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:53.197568 31025 external_mini_cluster.cc:1428] Setting key cbbac39034e9e6250a3bec56b618e281
I20260808 15:55:53.201931 26915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882: Bootstrap starting.
I20260808 15:55:53.204181 26915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:53.204926 26915 log.cc:824] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:53.206789 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.198980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.30.76.126:41467)
0808 15:55:53.199684 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.199712 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.199748 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:55:53.200382 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.200398 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.200771 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.201148 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:53.201571 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.202581 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.203488 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.204015 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.204197 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.205086 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:53.205102 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:53.205124 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:53.205504 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:53.205556 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:53.205575 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:53.205696 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.205848 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.206092 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":145,"threads_started":1}
I20260808 15:55:53.206919 26915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882: No bootstrap required, opened a new log
I20260808 15:55:53.210958 26915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e46f63c4048a29a673198454fc882" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41467 } }
I20260808 15:55:53.211215 26915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:53.211301 26915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 029e46f63c4048a29a673198454fc882, State: Initialized, Role: FOLLOWER
I20260808 15:55:53.211827 26915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e46f63c4048a29a673198454fc882" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41467 } }
I20260808 15:55:53.211993 26915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:53.212069 26915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:53.212167 26915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:53.213088 26915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e46f63c4048a29a673198454fc882" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41467 } }
I20260808 15:55:53.213438 26915 leader_election.cc:302] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 029e46f63c4048a29a673198454fc882; no voters:
I20260808 15:55:53.213774 26915 leader_election.cc:288] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:53.213905 26922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:53.214143 26922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [term 1 LEADER]: Becoming Leader. State: Replica: 029e46f63c4048a29a673198454fc882, State: Running, Role: LEADER
I20260808 15:55:53.214454 26922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e46f63c4048a29a673198454fc882" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41467 } }
I20260808 15:55:53.215026 26915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:53.216172 26926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 029e46f63c4048a29a673198454fc882. Latest consensus state: current_term: 1 leader_uuid: "029e46f63c4048a29a673198454fc882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e46f63c4048a29a673198454fc882" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41467 } } }
I20260808 15:55:53.216172 26924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "029e46f63c4048a29a673198454fc882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e46f63c4048a29a673198454fc882" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 41467 } } }
I20260808 15:55:53.216324 26926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:53.216324 26924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:53.216733 26937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:53.219730 26937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:53.225183 26937 catalog_manager.cc:1347] Generated new cluster ID: 8754a7514a84492287936ff705917618
I20260808 15:55:53.225256 26937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:53.237601 26937 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:53.238480 26937 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:53.245177 26937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882: Generated new TSK 0
I20260808 15:55:53.245913 26937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:55:53.250797 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.65:0
--local_ip_for_outbound_sockets=127.30.76.65
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41467
--builtin_ntp_servers=127.30.76.84:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:55:53.366672 26951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:53.366950 26951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:53.367025 26951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:53.371058 26951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:53.371129 26951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:53.371222 26951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.65
I20260808 15:55:53.376155 26951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.76.65
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.65
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:53.377184 26951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:53.378576 26951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:53.388124 26958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:53.388131 26965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:53.388226 26957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:53.388525 26951 server_base.cc:1034] running on GCE node
I20260808 15:55:53.388983 26951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:53.389585 26951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:53.390791 26951 hybrid_clock.cc:648] HybridClock initialized: now 1786204553390779 us; error 24 us; skew 500 ppm
I20260808 15:55:53.392556 26951 webserver.cc:460] Webserver started at http://127.30.76.65:39169/ using document root <none> and password file <none>
I20260808 15:55:53.393147 26951 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:53.393249 26951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:53.393488 26951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:53.395444 26951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data/instance:
uuid: "8f6ceb0da70643089bbdb90cbc063729"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "72656ae576c2541d6603e92bc6f8e533"
server_key_iv: "8c0508750ff5c7220e9da60ab6924484"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.396068 26951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance:
uuid: "8f6ceb0da70643089bbdb90cbc063729"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "72656ae576c2541d6603e92bc6f8e533"
server_key_iv: "8c0508750ff5c7220e9da60ab6924484"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.399507 26951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:55:53.401901 26977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.403052 26951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:55:53.403201 26951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal
uuid: "8f6ceb0da70643089bbdb90cbc063729"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "72656ae576c2541d6603e92bc6f8e533"
server_key_iv: "8c0508750ff5c7220e9da60ab6924484"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.403316 26951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:53.438500 26951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:53.439261 26951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:53.439486 26951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:53.440179 26951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:53.441205 26951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:53.441284 26951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.441359 26951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:53.441413 26951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.450295 26951 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.65:34395
I20260808 15:55:53.450323 27135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.65:34395 every 8 connection(s)
I20260808 15:55:53.451218 26951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/data/info.pb
I20260808 15:55:53.458592 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 26951
I20260808 15:55:53.458734 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-0/wal/instance
I20260808 15:55:53.459065 31025 external_mini_cluster.cc:1428] Setting key 584f40cf5ce87e374c29c301ecd2cf19
I20260808 15:55:53.460760 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.453143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.65:34463 (local address 127.30.76.126:41467)
0808 15:55:53.453298 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.453302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.454344 (+ 1042us) server_negotiation.cc:407] Connection header received
0808 15:55:53.455773 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.455776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.455829 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.455905 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:53.456771 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.457292 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.458390 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.458724 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.458836 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.460232 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:53.460235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:53.460237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:53.460250 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:53.460269 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:53.460277 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:53.460331 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.460543 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.460611 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:55:53.461352 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.66:0
--local_ip_for_outbound_sockets=127.30.76.66
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41467
--builtin_ntp_servers=127.30.76.84:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:55:53.461776 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.453546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41467 (local address 127.30.76.65:34463)
0808 15:55:53.454193 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:53.454233 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:53.455486 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:53.456075 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:53.456082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:53.456163 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:53.456632 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.456656 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.457404 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.457406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.458260 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.458269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.458908 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.458913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.459090 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.459930 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:53.459932 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:53.459983 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:53.460059 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:53.460368 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:53.460378 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:53.460384 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:55:53.460514 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 15:55:53.460855 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260808 15:55:53.463029 27136 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41467
I20260808 15:55:53.463326 27136 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:53.463928 27136 heartbeater.cc:511] Master 127.30.76.126:41467 requested a full tablet report, sending...
I20260808 15:55:53.465428 26867 ts_manager.cc:194] Registered new tserver with Master: 8f6ceb0da70643089bbdb90cbc063729 (127.30.76.65:34395)
I20260808 15:55:53.466580 26867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.65:34463
W20260808 15:55:53.574347 27140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:53.574572 27140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:53.574607 27140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:53.578579 27140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:53.578627 27140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:53.578718 27140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.66
I20260808 15:55:53.583340 27140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.76.66
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.66
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:53.584352 27140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:53.585654 27140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:53.595757 27151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:53.595738 27147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:53.595770 27148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:53.596129 27140 server_base.cc:1034] running on GCE node
I20260808 15:55:53.596539 27140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:53.597225 27140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:53.598423 27140 hybrid_clock.cc:648] HybridClock initialized: now 1786204553598401 us; error 44 us; skew 500 ppm
I20260808 15:55:53.600283 27140 webserver.cc:460] Webserver started at http://127.30.76.66:38153/ using document root <none> and password file <none>
I20260808 15:55:53.600844 27140 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:53.600960 27140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:53.601204 27140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:53.603116 27140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data/instance:
uuid: "f3b720e6cba746a59e1944ddffc94f2f"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "d035621091cf71525917792fe4ad05b1"
server_key_iv: "0a1ca0cf4823dfb93410aefe2dccb58d"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.603772 27140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance:
uuid: "f3b720e6cba746a59e1944ddffc94f2f"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "d035621091cf71525917792fe4ad05b1"
server_key_iv: "0a1ca0cf4823dfb93410aefe2dccb58d"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.607328 27140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260808 15:55:53.609714 27160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.610852 27140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:53.611021 27140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal
uuid: "f3b720e6cba746a59e1944ddffc94f2f"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "d035621091cf71525917792fe4ad05b1"
server_key_iv: "0a1ca0cf4823dfb93410aefe2dccb58d"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.611146 27140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:53.650800 27140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:53.651507 27140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:53.651750 27140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:53.652375 27140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:53.653359 27140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:53.653438 27140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.653515 27140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:53.653566 27140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.662355 27140 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.66:36605
I20260808 15:55:53.662395 27321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.66:36605 every 8 connection(s)
I20260808 15:55:53.663255 27140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/data/info.pb
I20260808 15:55:53.669476 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 27140
I20260808 15:55:53.669574 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal/instance
I20260808 15:55:53.669929 31025 external_mini_cluster.cc:1428] Setting key fa1f483abbe55b78733d5305ce872f9b
I20260808 15:55:53.671905 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.76.67:0
--local_ip_for_outbound_sockets=127.30.76.67
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41467
--builtin_ntp_servers=127.30.76.84:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:55:53.672458 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.664972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:60567 (local address 127.30.76.126:41467)
0808 15:55:53.665156 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.665161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.666245 (+ 1084us) server_negotiation.cc:407] Connection header received
0808 15:55:53.667581 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.667585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.667639 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.667712 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:53.668562 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.669070 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.670367 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.670586 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.670714 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.671925 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:53.671929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:53.671931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:53.671942 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:53.671960 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:53.671969 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:53.672056 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.672263 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.672321 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 15:55:53.673297 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.665382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41467 (local address 127.30.76.66:60567)
0808 15:55:53.666092 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:53.666132 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:53.667355 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:53.667871 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:53.667877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:53.667954 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:53.668410 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.668421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.669180 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.669183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.670232 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.670241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.670754 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.670759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.670917 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.671621 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:53.671623 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:53.671668 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:53.671728 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:53.672101 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:53.672115 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:53.672119 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:55:53.672247 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 15:55:53.672506 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260808 15:55:53.674414 27322 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41467
I20260808 15:55:53.674647 27322 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:53.675127 27322 heartbeater.cc:511] Master 127.30.76.126:41467 requested a full tablet report, sending...
I20260808 15:55:53.676221 26867 ts_manager.cc:194] Registered new tserver with Master: f3b720e6cba746a59e1944ddffc94f2f (127.30.76.66:36605)
I20260808 15:55:53.676775 26867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.66:60567
W20260808 15:55:53.787707 27325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:53.787969 27325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:53.788015 27325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:53.792237 27325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:53.792341 27325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:53.792448 27325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.76.67
I20260808 15:55:53.797240 27325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.76.67
--webserver_port=0
--tserver_master_addrs=127.30.76.126:41467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.76.67
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:53.798339 27325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:53.799844 27325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:53.810204 27335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:53.810343 27325 server_base.cc:1034] running on GCE node
W20260808 15:55:53.810204 27336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:53.810338 27339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:53.810920 27325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:53.811582 27325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:53.812759 27325 hybrid_clock.cc:648] HybridClock initialized: now 1786204553812704 us; error 72 us; skew 500 ppm
I20260808 15:55:53.814503 27325 webserver.cc:460] Webserver started at http://127.30.76.67:40185/ using document root <none> and password file <none>
I20260808 15:55:53.815128 27325 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:53.815197 27325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:53.815366 27325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:53.817590 27325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data/instance:
uuid: "0e5d004abe2f4cdab259e06d0c43b21a"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "8c2dac4e564528ffaef72878356e2b36"
server_key_iv: "5b48ccc9868ebaf94d17d53600d4e82e"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.818099 27325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance:
uuid: "0e5d004abe2f4cdab259e06d0c43b21a"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "8c2dac4e564528ffaef72878356e2b36"
server_key_iv: "5b48ccc9868ebaf94d17d53600d4e82e"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.821698 27325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 15:55:53.824146 27349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.825325 27325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:53.825424 27325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal
uuid: "0e5d004abe2f4cdab259e06d0c43b21a"
format_stamp: "Formatted at 2026-08-08 15:55:53 on dist-test-slave-sb2z"
server_key: "8c2dac4e564528ffaef72878356e2b36"
server_key_iv: "5b48ccc9868ebaf94d17d53600d4e82e"
server_key_version: "encryptionkey@0"
I20260808 15:55:53.825507 27325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:53.840492 27325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:53.841126 27325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:53.841279 27325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:53.841802 27325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:55:53.842675 27325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:55:53.842731 27325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.842772 27325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:55:53.842792 27325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:53.852706 27325 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.67:36707
I20260808 15:55:53.852730 27504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.67:36707 every 8 connection(s)
I20260808 15:55:53.853914 27325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/data/info.pb
I20260808 15:55:53.859814 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 27325
I20260808 15:55:53.859928 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal/instance
I20260808 15:55:53.860236 31025 external_mini_cluster.cc:1428] Setting key a60786647c6f02d584dd02521f44011c
I20260808 15:55:53.864010 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.855812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.67:51797 (local address 127.30.76.126:41467)
0808 15:55:53.855981 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.855986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.857042 (+ 1056us) server_negotiation.cc:407] Connection header received
0808 15:55:53.858397 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.858400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.858446 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.858520 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:53.859356 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.860119 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.861471 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.861771 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.861872 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.863448 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:53.863452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:53.863455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:53.863468 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:53.863489 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:53.863499 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:53.863622 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.863794 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.863854 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:55:53.864789 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.856232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:41467 (local address 127.30.76.67:51797)
0808 15:55:53.856880 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:53.856922 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:53.858211 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:53.858687 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:53.858693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:53.858771 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:55:53.859228 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.859240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.860267 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.860270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.861350 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.861360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.861936 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.861941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.862131 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.863148 (+ 1017us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:55:53.863150 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:55:53.863188 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:55:53.863252 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:55:53.863625 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:55:53.863637 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:55:53.863645 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:55:53.863794 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 15:55:53.864055 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":119,"threads_started":1}
I20260808 15:55:53.865885 27507 heartbeater.cc:347] Connected to a master server at 127.30.76.126:41467
I20260808 15:55:53.866181 27507 heartbeater.cc:464] Registering TS with master...
I20260808 15:55:53.866690 27507 heartbeater.cc:511] Master 127.30.76.126:41467 requested a full tablet report, sending...
I20260808 15:55:53.867763 26867 ts_manager.cc:194] Registered new tserver with Master: 0e5d004abe2f4cdab259e06d0c43b21a (127.30.76.67:36707)
I20260808 15:55:53.868297 26867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.76.67:51797
I20260808 15:55:53.874579 31025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:55:53.880035 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.876593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.30.76.126:41467)
0808 15:55:53.876730 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.876734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.876951 (+ 217us) server_negotiation.cc:407] Connection header received
0808 15:55:53.877052 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.877054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.877094 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.877170 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:53.877426 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.877892 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.878690 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.878887 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.878959 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.879697 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:53.879700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:53.879702 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:53.879713 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:53.879728 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:53.879736 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:53.879794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.879895 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.879939 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 15:55:53.882838 26867 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:55:53.885847 26867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:55:53.899832 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.894430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.30.76.66:36605)
0808 15:55:53.895049 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.895062 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.895282 (+ 220us) server_negotiation.cc:407] Connection header received
0808 15:55:53.895507 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.895513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.895707 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.895869 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:53.896064 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.896781 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.898546 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.899120 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.899254 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.899445 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.899579 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.899679 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":120,"threads_started":1}
I20260808 15:55:53.899930 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.893729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:34395 (local address 127.0.0.1:46022)
0808 15:55:53.894484 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:53.894535 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:53.894686 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:53.895551 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:53.895553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:53.895566 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:53.895736 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.895743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.897279 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.897286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.898388 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.898398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.899471 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.899474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.899592 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.899617 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:55:53.899684 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 15:55:53.899753 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":31,"thread_start_us":82,"threads_started":1}
I20260808 15:55:53.899825 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.894146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.66:36605 (local address 127.0.0.1:34742)
0808 15:55:53.895194 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:53.895206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:53.895344 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:53.895847 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:53.895850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:53.895869 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:53.895965 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.895974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.896954 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.896958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.898404 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.898415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.899255 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.899259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.899361 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.899397 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:55:53.899554 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 15:55:53.899674 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":70,"thread_start_us":90,"threads_started":1}
I20260808 15:55:53.900063 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.893828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36707 (local address 127.0.0.1:37032)
0808 15:55:53.894729 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:53.894743 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:53.894833 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:53.895499 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:53.895508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:53.895554 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:53.895736 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.895743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.897183 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.897187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.898531 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:53.898538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.899717 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.899721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:53.899796 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.899815 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 15:55:53.899879 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 15:55:53.899935 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":66,"spinlock_wait_cycles":2432,"thread_start_us":86,"threads_started":1}
I20260808 15:55:53.900393 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.893871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.30.76.65:34395)
0808 15:55:53.895063 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.895076 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.895103 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:55:53.895182 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.895187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.895388 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.895894 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:53.896029 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.897105 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.898517 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.899322 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.899901 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.900066 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.900120 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.900224 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":134,"threads_started":1}
I20260808 15:55:53.900596 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:53.894458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37032 (local address 127.30.76.67:36707)
0808 15:55:53.894976 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:53.894989 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:53.895014 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:55:53.895096 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:53.895104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:53.895315 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:53.895506 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:53.895876 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.897052 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.898647 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:53.899360 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:53.899459 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:53.899656 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:53.900330 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 15:55:53.900433 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":149,"threads_started":1}
I20260808 15:55:53.903417 27231 tablet_service.cc:1467] Processing CreateTablet for tablet b79b242a5f9a4354b1acbc2761bbf6d2 (DEFAULT_TABLE table=test-table [id=412286eda12b4c9a989482efa049032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:53.903417 27046 tablet_service.cc:1467] Processing CreateTablet for tablet b79b242a5f9a4354b1acbc2761bbf6d2 (DEFAULT_TABLE table=test-table [id=412286eda12b4c9a989482efa049032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:53.904573 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b79b242a5f9a4354b1acbc2761bbf6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:53.904573 27231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b79b242a5f9a4354b1acbc2761bbf6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:53.906008 27415 tablet_service.cc:1467] Processing CreateTablet for tablet b79b242a5f9a4354b1acbc2761bbf6d2 (DEFAULT_TABLE table=test-table [id=412286eda12b4c9a989482efa049032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:55:53.906971 27415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b79b242a5f9a4354b1acbc2761bbf6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:53.912251 27539 tablet_bootstrap.cc:492] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Bootstrap starting.
I20260808 15:55:53.913231 27541 tablet_bootstrap.cc:492] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Bootstrap starting.
I20260808 15:55:53.913417 27538 tablet_bootstrap.cc:492] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: Bootstrap starting.
I20260808 15:55:53.914757 27539 tablet_bootstrap.cc:654] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:53.915349 27541 tablet_bootstrap.cc:654] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:53.915473 27538 tablet_bootstrap.cc:654] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:53.915822 27539 log.cc:824] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:53.916249 27541 log.cc:824] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:53.916301 27538 log.cc:824] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:53.917991 27539 tablet_bootstrap.cc:492] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: No bootstrap required, opened a new log
I20260808 15:55:53.918198 27539 ts_tablet_manager.cc:1397] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:55:53.918310 27538 tablet_bootstrap.cc:492] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: No bootstrap required, opened a new log
I20260808 15:55:53.918314 27541 tablet_bootstrap.cc:492] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: No bootstrap required, opened a new log
I20260808 15:55:53.918435 27541 ts_tablet_manager.cc:1397] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260808 15:55:53.918488 27538 ts_tablet_manager.cc:1397] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:55:53.921551 27541 raft_consensus.cc:348] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:53.921794 27541 raft_consensus.cc:374] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:53.921849 27541 raft_consensus.cc:729] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5d004abe2f4cdab259e06d0c43b21a, State: Initialized, Role: FOLLOWER
I20260808 15:55:53.922307 27541 consensus_queue.cc:260] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:53.922451 27538 raft_consensus.cc:348] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:53.922719 27538 raft_consensus.cc:374] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:53.922786 27538 raft_consensus.cc:729] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6ceb0da70643089bbdb90cbc063729, State: Initialized, Role: FOLLOWER
I20260808 15:55:53.923089 27507 heartbeater.cc:503] Master 127.30.76.126:41467 was elected leader, sending a full tablet report...
I20260808 15:55:53.923336 27541 ts_tablet_manager.cc:1428] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260808 15:55:53.923430 27538 consensus_queue.cc:260] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:53.923370 27539 raft_consensus.cc:348] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:53.923622 27539 raft_consensus.cc:374] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:53.923699 27539 raft_consensus.cc:729] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3b720e6cba746a59e1944ddffc94f2f, State: Initialized, Role: FOLLOWER
I20260808 15:55:53.924149 27136 heartbeater.cc:503] Master 127.30.76.126:41467 was elected leader, sending a full tablet report...
I20260808 15:55:53.924121 27539 consensus_queue.cc:260] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:53.924386 27538 ts_tablet_manager.cc:1428] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 15:55:53.924896 27539 ts_tablet_manager.cc:1428] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 15:55:53.925853 27322 heartbeater.cc:503] Master 127.30.76.126:41467 was elected leader, sending a full tablet report...
W20260808 15:55:53.952421 27137 tablet.cc:2369] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:55:54.105018 27508 tablet.cc:2369] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:54.130810 27548 raft_consensus.cc:482] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:55:54.130998 27548 raft_consensus.cc:504] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:54.132191 27548 leader_election.cc:288] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f6ceb0da70643089bbdb90cbc063729 (127.30.76.65:34395), 0e5d004abe2f4cdab259e06d0c43b21a (127.30.76.67:36707)
I20260808 15:55:54.135829 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:54.132416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:34395 (local address 127.30.76.66:35387)
0808 15:55:54.132609 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:54.132624 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:54.132717 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:54.133013 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:54.133015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:54.133033 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:54.133157 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:54.133163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.133898 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.133903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:54.134741 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:54.134750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.135372 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.135375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:54.135474 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:54.135503 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 15:55:54.135622 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 15:55:54.135696 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":51}
I20260808 15:55:54.135830 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:54.132539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:35387 (local address 127.30.76.65:34395)
0808 15:55:54.132685 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:54.132689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:54.132705 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:55:54.132855 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:54.132858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:54.132903 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:54.132989 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:54.133278 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.133782 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.134869 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.135245 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.135382 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:54.135434 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:54.135650 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 15:55:54.135719 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:55:54.135986 27549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:54.132462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.67:36707 (local address 127.30.76.66:56777)
0808 15:55:54.132778 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:54.132789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 15:55:54.132912 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:55:54.133203 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:55:54.133206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:55:54.133223 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:55:54.133360 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:54.133365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.134082 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.134085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:54.134865 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:55:54.134874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.135563 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.135567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:55:54.135661 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:54.135693 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:55:54.135769 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 15:55:54.135842 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260808 15:55:54.135977 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:54.132513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.76.66:56777 (local address 127.30.76.67:36707)
0808 15:55:54.132695 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:54.132700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:54.132869 (+ 169us) server_negotiation.cc:407] Connection header received
0808 15:55:54.133033 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:54.133036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:54.133087 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:54.133188 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:55:54.133474 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.133955 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.134994 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.135401 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.135597 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:54.135659 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:54.135815 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 15:55:54.135878 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 15:55:54.136524 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79b242a5f9a4354b1acbc2761bbf6d2" candidate_uuid: "f3b720e6cba746a59e1944ddffc94f2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f6ceb0da70643089bbdb90cbc063729" is_pre_election: true
I20260808 15:55:54.136538 27442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79b242a5f9a4354b1acbc2761bbf6d2" candidate_uuid: "f3b720e6cba746a59e1944ddffc94f2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" is_pre_election: true
I20260808 15:55:54.136831 27442 raft_consensus.cc:2454] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3b720e6cba746a59e1944ddffc94f2f in term 0.
I20260808 15:55:54.136833 27070 raft_consensus.cc:2454] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3b720e6cba746a59e1944ddffc94f2f in term 0.
I20260808 15:55:54.137375 27166 leader_election.cc:302] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e5d004abe2f4cdab259e06d0c43b21a, 8f6ceb0da70643089bbdb90cbc063729, f3b720e6cba746a59e1944ddffc94f2f; no voters:
I20260808 15:55:54.137651 27548 raft_consensus.cc:2781] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:55:54.137738 27548 raft_consensus.cc:482] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:55:54.137784 27548 raft_consensus.cc:3037] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:54.138784 27548 raft_consensus.cc:504] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:54.139103 27548 leader_election.cc:288] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [CANDIDATE]: Term 1 election: Requested vote from peers 8f6ceb0da70643089bbdb90cbc063729 (127.30.76.65:34395), 0e5d004abe2f4cdab259e06d0c43b21a (127.30.76.67:36707)
I20260808 15:55:54.139432 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79b242a5f9a4354b1acbc2761bbf6d2" candidate_uuid: "f3b720e6cba746a59e1944ddffc94f2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f6ceb0da70643089bbdb90cbc063729"
I20260808 15:55:54.139459 27442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79b242a5f9a4354b1acbc2761bbf6d2" candidate_uuid: "f3b720e6cba746a59e1944ddffc94f2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5d004abe2f4cdab259e06d0c43b21a"
I20260808 15:55:54.139578 27442 raft_consensus.cc:3037] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:54.139601 27070 raft_consensus.cc:3037] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:54.140894 27442 raft_consensus.cc:2454] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3b720e6cba746a59e1944ddffc94f2f in term 1.
I20260808 15:55:54.140894 27070 raft_consensus.cc:2454] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3b720e6cba746a59e1944ddffc94f2f in term 1.
I20260808 15:55:54.141223 27161 leader_election.cc:302] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f6ceb0da70643089bbdb90cbc063729, f3b720e6cba746a59e1944ddffc94f2f; no voters:
I20260808 15:55:54.141427 27548 raft_consensus.cc:2781] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:54.141677 27548 raft_consensus.cc:686] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 1 LEADER]: Becoming Leader. State: Replica: f3b720e6cba746a59e1944ddffc94f2f, State: Running, Role: LEADER
I20260808 15:55:54.142025 27548 consensus_queue.cc:237] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } }
I20260808 15:55:54.145123 26866 catalog_manager.cc:5495] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f reported cstate change: term changed from 0 to 1, leader changed from <none> to f3b720e6cba746a59e1944ddffc94f2f (127.30.76.66). New cstate: current_term: 1 leader_uuid: "f3b720e6cba746a59e1944ddffc94f2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3b720e6cba746a59e1944ddffc94f2f" member_type: VOTER last_known_addr { host: "127.30.76.66" port: 36605 } health_report { overall_health: HEALTHY } } }
W20260808 15:55:54.164510 27323 tablet.cc:2369] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:55:54.189941 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:54.186572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.30.76.66:36605)
0808 15:55:54.186736 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:54.186740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:54.186753 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:55:54.186825 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:54.186827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:54.186881 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:54.186969 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:55:54.187205 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.187776 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.188590 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:54.188802 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:54.188879 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:54.189141 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:55:54.189540 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:55:54.189639 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:54.189690 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 15:55:54.189735 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 15:55:54.195772 27442 raft_consensus.cc:1261] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 1 FOLLOWER]: Refusing update from remote peer f3b720e6cba746a59e1944ddffc94f2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:54.195842 27070 raft_consensus.cc:1261] T b79b242a5f9a4354b1acbc2761bbf6d2 P 8f6ceb0da70643089bbdb90cbc063729 [term 1 FOLLOWER]: Refusing update from remote peer f3b720e6cba746a59e1944ddffc94f2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:55:54.196511 27548 consensus_queue.cc:1035] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f6ceb0da70643089bbdb90cbc063729" member_type: VOTER last_known_addr { host: "127.30.76.65" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:54.196691 27550 consensus_queue.cc:1035] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e5d004abe2f4cdab259e06d0c43b21a" member_type: VOTER last_known_addr { host: "127.30.76.67" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:55:54.206363 27558 mvcc.cc:205] Tried to move back new op lower bound from 7316293853978816512 to 7316293853768269824. Current Snapshot: MvccSnapshot[applied={T|T < 7316293853978816512}]
I20260808 15:55:54.209759 27559 mvcc.cc:205] Tried to move back new op lower bound from 7316293853978816512 to 7316293853768269824. Current Snapshot: MvccSnapshot[applied={T|T < 7316293853978816512}]
I20260808 15:55:54.213726 26866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:54.213905 26866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:55:54.216605 26866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 029e46f63c4048a29a673198454fc882: Sending DeleteTablet for 3 replicas of tablet b79b242a5f9a4354b1acbc2761bbf6d2
I20260808 15:55:54.217362 27046 tablet_service.cc:1514] Processing DeleteTablet for tablet b79b242a5f9a4354b1acbc2761bbf6d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:54 UTC) from {username='slave'} at 127.0.0.1:46022
I20260808 15:55:54.217432 27231 tablet_service.cc:1514] Processing DeleteTablet for tablet b79b242a5f9a4354b1acbc2761bbf6d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:54 UTC) from {username='slave'} at 127.0.0.1:34742
I20260808 15:55:54.217465 27415 tablet_service.cc:1514] Processing DeleteTablet for tablet b79b242a5f9a4354b1acbc2761bbf6d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:55:54 UTC) from {username='slave'} at 127.0.0.1:37032
I20260808 15:55:54.217952 27571 tablet_replica.cc:330] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: stopping tablet replica
I20260808 15:55:54.218050 27573 tablet_replica.cc:330] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: stopping tablet replica
I20260808 15:55:54.218369 27571 raft_consensus.cc:2229] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:55:54.218377 27573 raft_consensus.cc:2229] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:55:54.218691 27573 raft_consensus.cc:2258] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:54.218817 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 26951
I20260808 15:55:54.218854 27571 raft_consensus.cc:2258] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:55:54.220357 27573 ts_tablet_manager.cc:1905] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:54.220631 27571 ts_tablet_manager.cc:1905] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:55:54.223848 27571 ts_tablet_manager.cc:1918] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:54.223892 27573 ts_tablet_manager.cc:1918] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:55:54.223932 27571 log.cc:1196] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-1/wal/wals/b79b242a5f9a4354b1acbc2761bbf6d2
I20260808 15:55:54.224002 27573 log.cc:1196] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Deleting WAL directory at /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786204294980517-31025-0/minicluster-data/ts-2/wal/wals/b79b242a5f9a4354b1acbc2761bbf6d2
I20260808 15:55:54.224208 27571 ts_tablet_manager.cc:1939] T b79b242a5f9a4354b1acbc2761bbf6d2 P f3b720e6cba746a59e1944ddffc94f2f: Deleting consensus metadata
I20260808 15:55:54.224434 27573 ts_tablet_manager.cc:1939] T b79b242a5f9a4354b1acbc2761bbf6d2 P 0e5d004abe2f4cdab259e06d0c43b21a: Deleting consensus metadata
I20260808 15:55:54.226208 26844 catalog_manager.cc:4841] TS 0e5d004abe2f4cdab259e06d0c43b21a (127.30.76.67:36707): tablet b79b242a5f9a4354b1acbc2761bbf6d2 (table test-table [id=412286eda12b4c9a989482efa049032c]) successfully deleted
I20260808 15:55:54.231860 26841 catalog_manager.cc:4841] TS f3b720e6cba746a59e1944ddffc94f2f (127.30.76.66:36605): tablet b79b242a5f9a4354b1acbc2761bbf6d2 (table test-table [id=412286eda12b4c9a989482efa049032c]) successfully deleted
W20260808 15:55:54.234992 26840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.76.65:34395 (error 108)
I20260808 15:55:54.235966 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 27140
I20260808 15:55:54.236184 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:54.235852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.65:34395 (local address 127.0.0.1:46024)
0808 15:55:54.235981 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 15:55:54.236067 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.65:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 15:55:54.238400 26840 catalog_manager.cc:4584] TS 8f6ceb0da70643089bbdb90cbc063729 (127.30.76.65:34395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b79b242a5f9a4354b1acbc2761bbf6d2: Network error: Client connection negotiation failed: client connection to 127.30.76.65:34395: connect: Connection refused (error 111)
I20260808 15:55:54.245129 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 27325
I20260808 15:55:54.258750 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 26819
2026-08-08T15:55:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1272 ms)
[----------] 6 tests from EncryptionPolicyTest (11414 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:54 dist-test-slave-sb2z krb5kdc[27577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:54 dist-test-slave-sb2z krb5kdc[27577](info): set up 2 sockets
Aug 08 15:55:54 dist-test-slave-sb2z krb5kdc[27577](info): commencing operation
krb5kdc: starting...
W20260808 15:55:55.495329 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:55:55 dist-test-slave-sb2z krb5kdc[27577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:55:55.626493 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:55.741230 27591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:55.741526 27591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:55.741604 27591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:55.745628 27591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:55.745724 27591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:55.745764 27591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:55.745823 27591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:55.745865 27591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:55.751067 27591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:55.752215 27591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:55.753280 27591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:55.762130 27598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:55.762167 27601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:55.762403 27597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:55.762424 27591 server_base.cc:1034] running on GCE node
I20260808 15:55:55.763191 27591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:55.764374 27591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:55.765560 27591 hybrid_clock.cc:648] HybridClock initialized: now 1786204555765525 us; error 49 us; skew 500 ppm
Aug 08 15:55:55 dist-test-slave-sb2z krb5kdc[27577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:55.768360 27591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:55:55.769330 27591 webserver.cc:460] Webserver started at http://127.0.0.1:45391/ using document root <none> and password file <none>
I20260808 15:55:55.769907 27591 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:55.770005 27591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:55.770251 27591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:55.772468 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "a4bf8f46cd6049b799d7978e4219c7bc"
format_stamp: "Formatted at 2026-08-08 15:55:55 on dist-test-slave-sb2z"
server_key: "f4f5b7c642d10b093660fd05e117a2ec"
server_key_iv: "d3bd5a563d0bae76bdf4f06afc7e402d"
server_key_version: "encryptionkey@0"
I20260808 15:55:55.773051 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "a4bf8f46cd6049b799d7978e4219c7bc"
format_stamp: "Formatted at 2026-08-08 15:55:55 on dist-test-slave-sb2z"
server_key: "f4f5b7c642d10b093660fd05e117a2ec"
server_key_iv: "d3bd5a563d0bae76bdf4f06afc7e402d"
server_key_version: "encryptionkey@0"
I20260808 15:55:55.776731 27591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:55:55.779008 27611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:55.780208 27591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:55:55.780364 27591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "a4bf8f46cd6049b799d7978e4219c7bc"
format_stamp: "Formatted at 2026-08-08 15:55:55 on dist-test-slave-sb2z"
server_key: "f4f5b7c642d10b093660fd05e117a2ec"
server_key_iv: "d3bd5a563d0bae76bdf4f06afc7e402d"
server_key_version: "encryptionkey@0"
I20260808 15:55:55.780478 27591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:55.801021 27591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:55.803795 27591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:55.803997 27591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:55.811250 27591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38131
I20260808 15:55:55.811280 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38131 every 8 connection(s)
I20260808 15:55:55.812214 27591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:55.813164 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 27591
I20260808 15:55:55.813313 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:55.813656 31025 external_mini_cluster.cc:1428] Setting key dedf9dec68fb21231c4ad72fcb3d88c6
I20260808 15:55:55.815490 27684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:55.821135 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc: Bootstrap starting.
Aug 08 15:55:55 dist-test-slave-sb2z krb5kdc[27577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:55:55.823693 27684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:55.824432 27684 log.cc:824] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:55.826298 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc: No bootstrap required, opened a new log
I20260808 15:55:55.828150 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:55.815109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59029 (local address 127.0.0.1:38131)
0808 15:55:55.815651 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:55.815667 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:55.815705 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:55:55.816327 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:55.816342 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:55.816673 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:55.817003 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:55.817844 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:55.818839 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:55.819751 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:55.820206 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:55.820372 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:55.823212 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:55.823217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:55.823229 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:55.823256 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:55.825063 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:55.825585 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:55.825590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:55.825594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:55.825715 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:55.826068 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:55.826075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:55.826077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:55.826506 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:55.826740 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:55.827065 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 15:55:55.827299 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":151,"threads_started":1}
I20260808 15:55:55.830539 27684 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260808 15:55:55.830813 27684 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:55.830889 27684 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4bf8f46cd6049b799d7978e4219c7bc, State: Initialized, Role: FOLLOWER
I20260808 15:55:55.831331 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:55:55.831352 27684 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260808 15:55:55.831571 27684 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:55.831642 27684 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:55.831740 27684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:55.832691 27684 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260808 15:55:55.833036 27684 leader_election.cc:302] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4bf8f46cd6049b799d7978e4219c7bc; no voters:
I20260808 15:55:55.833389 27684 leader_election.cc:288] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:55.833499 27697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:55.833725 27697 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [term 1 LEADER]: Becoming Leader. State: Replica: a4bf8f46cd6049b799d7978e4219c7bc, State: Running, Role: LEADER
I20260808 15:55:55.834034 27697 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260808 15:55:55.834758 27684 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:55.835625 27703 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4bf8f46cd6049b799d7978e4219c7bc. Latest consensus state: current_term: 1 leader_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } } }
I20260808 15:55:55.835659 27699 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bf8f46cd6049b799d7978e4219c7bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } } }
I20260808 15:55:55.835754 27703 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:55.835768 27699 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:55.836268 27705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:55.840298 27705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:55.843662 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:55.833454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55199 (local address 127.0.0.1:38131)
0808 15:55:55.833567 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:55.833571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:55.833841 (+ 270us) server_negotiation.cc:407] Connection header received
0808 15:55:55.834072 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:55.834075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:55.834119 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:55.834167 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:55.835156 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:55.836357 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:55.837475 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:55.837802 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:55.837877 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:55.839729 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:55.839733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:55.839736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:55:55.839761 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:55.841490 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:55.842081 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:55.842084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:55.842086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:55.842178 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:55:55.842578 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:55:55.842582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:55:55.842583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:55:55.842804 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:55.842927 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:55.843363 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 15:55:55.843458 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 15:55:55.846864 27705 catalog_manager.cc:1347] Generated new cluster ID: 17eea4dc36dc4952842127a34b7b9a25
I20260808 15:55:55.846927 27705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:55.856498 27705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:55.857625 27705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:55.867095 27705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4bf8f46cd6049b799d7978e4219c7bc: Generated new TSK 0
I20260808 15:55:55.867831 27705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:55:55.876856 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 27591
W20260808 15:55:55.889746 27575 connection.cc:535] client connection to 127.0.0.1:38131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38131): Connection reset by peer (error 104)
2026-08-08T15:55:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1619 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:55 dist-test-slave-sb2z krb5kdc[27736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:55 dist-test-slave-sb2z krb5kdc[27736](info): set up 2 sockets
Aug 08 15:55:55 dist-test-slave-sb2z krb5kdc[27736](info): commencing operation
krb5kdc: starting...
W20260808 15:55:57.078253 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:55:57 dist-test-slave-sb2z krb5kdc[27736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:55:57.199834 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:57.310478 27756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:57.310774 27756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:57.310861 27756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:57.314836 27756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:57.314920 27756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:57.314961 27756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:57.315029 27756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:57.315068 27756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:57.322947 27756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:57.324069 27756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:57.325099 27756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:55:57.333688 27756 server_base.cc:1034] running on GCE node
W20260808 15:55:57.333807 27763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:57.333667 27761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:57.333742 27766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:57.334434 27756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:57.335479 27756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:57.336695 27756 hybrid_clock.cc:648] HybridClock initialized: now 1786204557336667 us; error 38 us; skew 500 ppm
Aug 08 15:55:57 dist-test-slave-sb2z krb5kdc[27736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:57.343396 27756 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:55:57.344368 27756 webserver.cc:460] Webserver started at http://127.0.0.1:41763/ using document root <none> and password file <none>
I20260808 15:55:57.344928 27756 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:57.345031 27756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:57.345269 27756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:57.347265 27756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6"
format_stamp: "Formatted at 2026-08-08 15:55:57 on dist-test-slave-sb2z"
server_key: "5caee6e77e425993c293f2d5c01c11b9"
server_key_iv: "a5443f24375cb5e2022a49edcbae9dd0"
server_key_version: "encryptionkey@0"
I20260808 15:55:57.347851 27756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6"
format_stamp: "Formatted at 2026-08-08 15:55:57 on dist-test-slave-sb2z"
server_key: "5caee6e77e425993c293f2d5c01c11b9"
server_key_iv: "a5443f24375cb5e2022a49edcbae9dd0"
server_key_version: "encryptionkey@0"
I20260808 15:55:57.351482 27756 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:55:57.353833 27773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:57.354980 27756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:57.355152 27756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6"
format_stamp: "Formatted at 2026-08-08 15:55:57 on dist-test-slave-sb2z"
server_key: "5caee6e77e425993c293f2d5c01c11b9"
server_key_iv: "a5443f24375cb5e2022a49edcbae9dd0"
server_key_version: "encryptionkey@0"
I20260808 15:55:57.355262 27756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:57.376360 27756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:57.379120 27756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:57.379320 27756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:57.386590 27756 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42981
I20260808 15:55:57.386602 27845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42981 every 8 connection(s)
I20260808 15:55:57.387606 27756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:57.390268 27846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:57.395555 27846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6: Bootstrap starting.
I20260808 15:55:57.396028 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 27756
I20260808 15:55:57.396184 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:57.396461 31025 external_mini_cluster.cc:1428] Setting key 7684cccd546873b9e8b9d8ffea363b93
I20260808 15:55:57.398232 27846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:57.398928 27846 log.cc:824] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:57.400949 27846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6: No bootstrap required, opened a new log
I20260808 15:55:57.404361 27846 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42981 } }
I20260808 15:55:57.404651 27846 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:57.404754 27846 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c70e30e3ea4539a44ed3cc2089a9a6, State: Initialized, Role: FOLLOWER
I20260808 15:55:57.405226 27846 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42981 } }
I20260808 15:55:57.405292 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:57.397742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32909 (local address 127.0.0.1:42981)
0808 15:55:57.398183 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:57.398198 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:57.398234 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:55:57.398905 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:57.398922 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:57.399328 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:57.399717 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:57.400038 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:57.400984 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:57.401898 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:57.402364 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:57.402533 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:57.403470 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:57.403475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:57.403486 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:57.403911 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:57.403964 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:57.403986 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:57.404137 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:57.404306 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 15:55:57.404434 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260808 15:55:57.405388 27846 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:57.405463 27846 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:57.405570 27846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:57.406558 27846 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42981 } }
I20260808 15:55:57.406885 27846 leader_election.cc:302] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c70e30e3ea4539a44ed3cc2089a9a6; no voters:
I20260808 15:55:57.407188 27846 leader_election.cc:288] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:57.407279 27856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:57.407568 27856 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [term 1 LEADER]: Becoming Leader. State: Replica: a3c70e30e3ea4539a44ed3cc2089a9a6, State: Running, Role: LEADER
I20260808 15:55:57.407940 27856 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42981 } }
I20260808 15:55:57.408334 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:55:57.408540 27846 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:57.411212 27860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c70e30e3ea4539a44ed3cc2089a9a6. Latest consensus state: current_term: 1 leader_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42981 } } }
I20260808 15:55:57.411077 27859 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c70e30e3ea4539a44ed3cc2089a9a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42981 } } }
I20260808 15:55:57.411471 27859 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:57.411764 27860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6 [sys.catalog]: This master's current role is: LEADER
W20260808 15:55:57.414022 27882 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:55:57.414140 27882 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:55:57.414505 27884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:57.415150 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:57.410939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40855 (local address 127.0.0.1:42981)
0808 15:55:57.411180 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:57.411184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:57.411268 (+ 84us) server_negotiation.cc:407] Connection header received
0808 15:55:57.411496 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:57.411499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:57.411568 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:57.411660 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:57.411996 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:57.412529 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:57.413343 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:55:57.413563 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:55:57.413677 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:55:57.414505 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:57.414509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:57.414512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:57.414527 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:57.414555 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:57.414568 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:57.414651 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:57.414962 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 15:55:57.415012 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260808 15:55:57.415429 27884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:57.420810 27884 catalog_manager.cc:1347] Generated new cluster ID: 47d767e65b194a5cb8af40a779a62abd
I20260808 15:55:57.420876 27884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:57.448499 27884 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:57.449502 27884 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:57.458825 27884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3c70e30e3ea4539a44ed3cc2089a9a6: Generated new TSK 0
I20260808 15:55:57.459643 27884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:55:57.464896 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 27756
W20260808 15:55:57.477823 27731 connection.cc:535] client connection to 127.0.0.1:42981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42981): Connection reset by peer (error 104)
2026-08-08T15:55:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:57 dist-test-slave-sb2z krb5kdc[27890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:57 dist-test-slave-sb2z krb5kdc[27890](info): set up 2 sockets
Aug 08 15:55:57 dist-test-slave-sb2z krb5kdc[27890](info): commencing operation
krb5kdc: starting...
W20260808 15:55:58.672448 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:55:58 dist-test-slave-sb2z krb5kdc[27890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:55:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:55:58.796464 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:55:58.907428 27905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:55:58.907706 27905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:55:58.907747 27905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:55:58.911563 27905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:55:58.911608 27905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:55:58.911633 27905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:55:58.911648 27905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:55:58.911669 27905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:55:58.916647 27905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:55:58.917550 27905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:55:58.918486 27905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:55:58.926865 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:58.927007 27905 server_base.cc:1034] running on GCE node
W20260808 15:55:58.926946 27915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:55:58.926947 27916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:55:58.927794 27905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:55:58.928910 27905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:55:58.930094 27905 hybrid_clock.cc:648] HybridClock initialized: now 1786204558930058 us; error 47 us; skew 500 ppm
Aug 08 15:55:58 dist-test-slave-sb2z krb5kdc[27890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204558, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:55:58.932813 27905 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:55:58.933683 27905 webserver.cc:460] Webserver started at http://127.0.0.1:35145/ using document root <none> and password file <none>
I20260808 15:55:58.934223 27905 fs_manager.cc:359] Metadata directory not provided
I20260808 15:55:58.934314 27905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:55:58.934542 27905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:55:58.936619 27905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "8fc978e471c24ea4b719eb1bfa0cce79"
format_stamp: "Formatted at 2026-08-08 15:55:58 on dist-test-slave-sb2z"
server_key: "7fdd5e2fab31bc30fc362a3876be88d7"
server_key_iv: "e12d4fe5ac76a67101461e64dc4e32ed"
server_key_version: "encryptionkey@0"
I20260808 15:55:58.937196 27905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "8fc978e471c24ea4b719eb1bfa0cce79"
format_stamp: "Formatted at 2026-08-08 15:55:58 on dist-test-slave-sb2z"
server_key: "7fdd5e2fab31bc30fc362a3876be88d7"
server_key_iv: "e12d4fe5ac76a67101461e64dc4e32ed"
server_key_version: "encryptionkey@0"
I20260808 15:55:58.940703 27905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:55:58.942952 27933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:55:58.944167 27905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:55:58.944309 27905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "8fc978e471c24ea4b719eb1bfa0cce79"
format_stamp: "Formatted at 2026-08-08 15:55:58 on dist-test-slave-sb2z"
server_key: "7fdd5e2fab31bc30fc362a3876be88d7"
server_key_iv: "e12d4fe5ac76a67101461e64dc4e32ed"
server_key_version: "encryptionkey@0"
I20260808 15:55:58.944418 27905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:55:58.948889 27905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:55:58.951740 27905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:55:58.951930 27905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:55:58.959035 27905 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46503
I20260808 15:55:58.959044 28008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46503 every 8 connection(s)
I20260808 15:55:58.960000 27905 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:55:58.962211 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 27905
I20260808 15:55:58.962370 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:55:58.962684 31025 external_mini_cluster.cc:1428] Setting key 55f77405811b961ad61c00125c94a2fd
I20260808 15:55:58.962962 28009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:55:58.967783 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:58.964015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47611 (local address 127.0.0.1:46503)
0808 15:55:58.964431 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:58.964446 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:58.964480 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:55:58.965094 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:58.965109 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:58.965371 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:58.965670 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:58.966525 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:58.966529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:58.966540 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:58.966550 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:58.966601 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:58.966619 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:58.966734 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:58.966860 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 15:55:58.967020 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":148,"threads_started":1}
I20260808 15:55:58.969836 28009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79: Bootstrap starting.
I20260808 15:55:58.970582 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:55:58.972568 28009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79: Neither blocks nor log segments found. Creating new log.
I20260808 15:55:58.973407 28009 log.cc:824] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79: Log is configured to *not* fsync() on all Append() calls
I20260808 15:55:58.974493 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:55:58.973096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45033 (local address 127.0.0.1:46503)
0808 15:55:58.973232 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 15:55:58.973236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:55:58.973257 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:55:58.973283 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:55:58.973285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:55:58.973307 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:55:58.973406 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:55:58.974085 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:55:58.974091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:55:58.974097 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:55:58.974114 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:55:58.974138 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:55:58.974153 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:55:58.974245 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 15:55:58.974341 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 15:55:58.974369 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:55:58.975659 28009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79: No bootstrap required, opened a new log
I20260808 15:55:58.978909 28009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46503 } }
I20260808 15:55:58.979133 28009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:55:58.979194 28009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc978e471c24ea4b719eb1bfa0cce79, State: Initialized, Role: FOLLOWER
I20260808 15:55:58.979621 28009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46503 } }
I20260808 15:55:58.979775 28009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:55:58.979849 28009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:55:58.979950 28009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:55:58.980829 28009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46503 } }
I20260808 15:55:58.981122 28009 leader_election.cc:302] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fc978e471c24ea4b719eb1bfa0cce79; no voters:
I20260808 15:55:58.981387 28009 leader_election.cc:288] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:55:58.981482 28024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:55:58.981716 28024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [term 1 LEADER]: Becoming Leader. State: Replica: 8fc978e471c24ea4b719eb1bfa0cce79, State: Running, Role: LEADER
I20260808 15:55:58.982045 28024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46503 } }
I20260808 15:55:58.982648 28009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:55:58.983649 28030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc978e471c24ea4b719eb1bfa0cce79. Latest consensus state: current_term: 1 leader_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46503 } } }
I20260808 15:55:58.983732 28028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc978e471c24ea4b719eb1bfa0cce79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46503 } } }
I20260808 15:55:58.983832 28030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:58.983920 28028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79 [sys.catalog]: This master's current role is: LEADER
I20260808 15:55:58.984345 28039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:55:58.986945 28039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:55:58.992246 28039 catalog_manager.cc:1347] Generated new cluster ID: 7a7888ec2db24f478f64cf8c5d04fafc
I20260808 15:55:58.992316 28039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:55:58.999063 28039 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:55:58.999341 28039 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:55:59.009241 28039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fc978e471c24ea4b719eb1bfa0cce79: Generated new TSK 0
I20260808 15:55:59.009899 28039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:55:59.017037 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 27905
2026-08-08T15:55:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:55:59 dist-test-slave-sb2z krb5kdc[28056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:55:59 dist-test-slave-sb2z krb5kdc[28056](info): set up 2 sockets
Aug 08 15:55:59 dist-test-slave-sb2z krb5kdc[28056](info): commencing operation
krb5kdc: starting...
W20260808 15:56:00.240954 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:00 dist-test-slave-sb2z krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:00.362382 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:00.473635 28070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:00.473953 28070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:00.474035 28070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:00.478012 28070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:00.478109 28070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:00.478148 28070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:00.478207 28070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:00.478247 28070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:00.483294 28070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:00.484320 28070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:00.485385 28070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:56:00.494064 28070 server_base.cc:1034] running on GCE node
W20260808 15:56:00.494052 28080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:00.494014 28076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:00.494014 28078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:00.494900 28070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:00.496002 28070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:00.497164 28070 hybrid_clock.cc:648] HybridClock initialized: now 1786204560497135 us; error 41 us; skew 500 ppm
Aug 08 15:56:00 dist-test-slave-sb2z krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:00.499966 28070 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:00.500799 28070 webserver.cc:460] Webserver started at http://127.0.0.1:41951/ using document root <none> and password file <none>
I20260808 15:56:00.501358 28070 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:00.501452 28070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:00.501703 28070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:00.503791 28070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "d1791f7171694dd2a96414bb0059e49e"
format_stamp: "Formatted at 2026-08-08 15:56:00 on dist-test-slave-sb2z"
server_key: "1736d51b9751122ea3101f0e587ecc80"
server_key_iv: "2eccbf5ff43d1d7dbf5fc33eac33e91e"
server_key_version: "encryptionkey@0"
I20260808 15:56:00.504347 28070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "d1791f7171694dd2a96414bb0059e49e"
format_stamp: "Formatted at 2026-08-08 15:56:00 on dist-test-slave-sb2z"
server_key: "1736d51b9751122ea3101f0e587ecc80"
server_key_iv: "2eccbf5ff43d1d7dbf5fc33eac33e91e"
server_key_version: "encryptionkey@0"
I20260808 15:56:00.507968 28070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:56:00.510310 28090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:00.511475 28070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:00.511649 28070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "d1791f7171694dd2a96414bb0059e49e"
format_stamp: "Formatted at 2026-08-08 15:56:00 on dist-test-slave-sb2z"
server_key: "1736d51b9751122ea3101f0e587ecc80"
server_key_iv: "2eccbf5ff43d1d7dbf5fc33eac33e91e"
server_key_version: "encryptionkey@0"
I20260808 15:56:00.511765 28070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:00.529456 28070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:00.532444 28070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:00.532657 28070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:00.540223 28070 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46703
I20260808 15:56:00.540252 28157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46703 every 8 connection(s)
I20260808 15:56:00.541183 28070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:00.543905 28158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:00.549419 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 28070
I20260808 15:56:00.549528 28158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e: Bootstrap starting.
I20260808 15:56:00.549582 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:00.549862 31025 external_mini_cluster.cc:1428] Setting key 3d1cff31bd7b3804893a35247254e6aa
I20260808 15:56:00.552122 28158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:00.552872 28158 log.cc:824] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:00.554721 28158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e: No bootstrap required, opened a new log
I20260808 15:56:00.558449 28158 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1791f7171694dd2a96414bb0059e49e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
Aug 08 15:56:00 dist-test-slave-sb2z krb5kdc[28056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:56:00.558728 28158 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:00.558813 28158 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1791f7171694dd2a96414bb0059e49e, State: Initialized, Role: FOLLOWER
I20260808 15:56:00.559384 28158 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1791f7171694dd2a96414bb0059e49e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260808 15:56:00.559569 28158 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:00.559653 28158 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:00.559767 28158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:00.561048 28158 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1791f7171694dd2a96414bb0059e49e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260808 15:56:00.561432 28158 leader_election.cc:302] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1791f7171694dd2a96414bb0059e49e; no voters:
I20260808 15:56:00.561787 28158 leader_election.cc:288] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:00.561879 28164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:00.562105 28164 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [term 1 LEADER]: Becoming Leader. State: Replica: d1791f7171694dd2a96414bb0059e49e, State: Running, Role: LEADER
I20260808 15:56:00.562832 28164 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1791f7171694dd2a96414bb0059e49e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260808 15:56:00.563146 28158 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:00.564530 28166 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1791f7171694dd2a96414bb0059e49e. Latest consensus state: current_term: 1 leader_uuid: "d1791f7171694dd2a96414bb0059e49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1791f7171694dd2a96414bb0059e49e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } } }
I20260808 15:56:00.564666 28166 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:00.565073 28165 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1791f7171694dd2a96414bb0059e49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1791f7171694dd2a96414bb0059e49e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } } }
I20260808 15:56:00.565148 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:00.551211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43777 (local address 127.0.0.1:46703)
0808 15:56:00.551703 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:00.551719 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:00.551755 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:56:00.552458 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:00.552478 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:00.552817 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:00.553147 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:00.553991 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:00.555030 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:00.555967 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:00.556431 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:00.556609 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:00.559402 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:00.559410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:00.559426 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:00.559457 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:00.561526 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:00.562069 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:00.562079 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:00.562088 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:00.562275 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:00.562613 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:00.562617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:00.562618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:00.563126 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:00.563335 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:00.563717 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 15:56:00.564073 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":133,"threads_started":1}
I20260808 15:56:00.565169 28165 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:00.565793 28178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:00.568275 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:00.569428 28178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:00.575424 28178 catalog_manager.cc:1347] Generated new cluster ID: edb45c0873a54aaeb993621a1f331e01
I20260808 15:56:00.575485 28178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:00.579150 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:00.570259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46907 (local address 127.0.0.1:46703)
0808 15:56:00.570353 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:00.570357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:00.570788 (+ 431us) server_negotiation.cc:407] Connection header received
0808 15:56:00.570936 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:00.570939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:00.570988 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:00.571077 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:00.571882 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:00.572721 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:00.573785 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:00.574031 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:00.574147 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:00.575694 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:00.575698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:00.575701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:00.575727 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:00.577424 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:00.577884 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:00.577888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:00.577890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:00.577982 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:00.578271 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:00.578274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:00.578276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:00.578499 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:00.578644 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:00.578867 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 15:56:00.578993 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 15:56:00.581025 28178 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:00.581868 28178 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:00.588747 28178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1791f7171694dd2a96414bb0059e49e: Generated new TSK 0
I20260808 15:56:00.589428 28178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:00.598390 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 28070
2026-08-08T15:56:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:00 dist-test-slave-sb2z krb5kdc[28209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:00 dist-test-slave-sb2z krb5kdc[28209](info): set up 2 sockets
Aug 08 15:56:00 dist-test-slave-sb2z krb5kdc[28209](info): commencing operation
krb5kdc: starting...
W20260808 15:56:01.910784 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:02 dist-test-slave-sb2z krb5kdc[28209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:02.086498 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:02.207926 28229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:02.208165 28229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:02.208237 28229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:02.212174 28229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:02.212227 28229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:02.212246 28229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:02.212263 28229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:02.212278 28229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:02.217389 28229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:02.218427 28229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:02.219470 28229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:02.227860 28238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:02.227854 28241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:02.228044 28229 server_base.cc:1034] running on GCE node
W20260808 15:56:02.227854 28236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:02.228746 28229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:02.229887 28229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:02.231076 28229 hybrid_clock.cc:648] HybridClock initialized: now 1786204562231036 us; error 47 us; skew 500 ppm
Aug 08 15:56:02 dist-test-slave-sb2z krb5kdc[28209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:02.233831 28229 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:02.234757 28229 webserver.cc:460] Webserver started at http://127.0.0.1:34463/ using document root <none> and password file <none>
I20260808 15:56:02.235318 28229 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:02.235409 28229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:02.235697 28229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:02.237733 28229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "b2a87353f4c349f7a58145b5f29be8a8"
format_stamp: "Formatted at 2026-08-08 15:56:02 on dist-test-slave-sb2z"
server_key: "8d9c8a17630c8cdcbfba9c0e151d9210"
server_key_iv: "36d8f7326805b05a825b856c0f7dd6bf"
server_key_version: "encryptionkey@0"
I20260808 15:56:02.238308 28229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "b2a87353f4c349f7a58145b5f29be8a8"
format_stamp: "Formatted at 2026-08-08 15:56:02 on dist-test-slave-sb2z"
server_key: "8d9c8a17630c8cdcbfba9c0e151d9210"
server_key_iv: "36d8f7326805b05a825b856c0f7dd6bf"
server_key_version: "encryptionkey@0"
I20260808 15:56:02.242080 28229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:56:02.244578 28249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:02.245734 28229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:56:02.245872 28229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "b2a87353f4c349f7a58145b5f29be8a8"
format_stamp: "Formatted at 2026-08-08 15:56:02 on dist-test-slave-sb2z"
server_key: "8d9c8a17630c8cdcbfba9c0e151d9210"
server_key_iv: "36d8f7326805b05a825b856c0f7dd6bf"
server_key_version: "encryptionkey@0"
I20260808 15:56:02.245980 28229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:02.273613 28229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:02.276412 28229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:02.276610 28229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:02.283696 28229 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46279
I20260808 15:56:02.283725 28324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46279 every 8 connection(s)
I20260808 15:56:02.284623 28229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:02.287230 28325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:02.292567 28325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8: Bootstrap starting.
I20260808 15:56:02.294330 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 28229
I20260808 15:56:02.294488 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:02.294782 31025 external_mini_cluster.cc:1428] Setting key a7b6a03d4926a6f69590b6243f37b83a
I20260808 15:56:02.295221 28325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:02.296118 28325 log.cc:824] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:02.297922 28325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8: No bootstrap required, opened a new log
I20260808 15:56:02.301378 28325 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a87353f4c349f7a58145b5f29be8a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260808 15:56:02.301610 28325 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:02.301697 28325 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a87353f4c349f7a58145b5f29be8a8, State: Initialized, Role: FOLLOWER
I20260808 15:56:02.302119 28325 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a87353f4c349f7a58145b5f29be8a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260808 15:56:02.302268 28325 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:02.302340 28325 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:02.302466 28325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:02.303318 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:02.295939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49395 (local address 127.0.0.1:46279)
0808 15:56:02.296423 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:02.296437 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:02.296470 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:56:02.297081 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:02.297096 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:02.297416 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:02.297745 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:02.298154 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:02.299074 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:02.299995 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:02.300460 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:02.300639 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:02.301464 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:02.301469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:02.301481 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:02.301964 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:02.302042 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:02.302066 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:02.302168 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:02.302319 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 15:56:02.302668 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":144,"threads_started":1}
I20260808 15:56:02.303328 28325 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a87353f4c349f7a58145b5f29be8a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260808 15:56:02.303764 28325 leader_election.cc:302] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2a87353f4c349f7a58145b5f29be8a8; no voters:
I20260808 15:56:02.304127 28325 leader_election.cc:288] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:02.304200 28334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:02.304452 28334 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [term 1 LEADER]: Becoming Leader. State: Replica: b2a87353f4c349f7a58145b5f29be8a8, State: Running, Role: LEADER
I20260808 15:56:02.304742 28334 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a87353f4c349f7a58145b5f29be8a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260808 15:56:02.305858 28334 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a87353f4c349f7a58145b5f29be8a8. Latest consensus state: current_term: 1 leader_uuid: "b2a87353f4c349f7a58145b5f29be8a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a87353f4c349f7a58145b5f29be8a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } } }
I20260808 15:56:02.306030 28334 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:02.306159 28325 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:02.306398 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:02.306370 28335 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2a87353f4c349f7a58145b5f29be8a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a87353f4c349f7a58145b5f29be8a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } } }
I20260808 15:56:02.306465 28335 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8 [sys.catalog]: This master's current role is: LEADER
W20260808 15:56:02.311426 28360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:02.311561 28360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:02.311650 28339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:02.312322 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:02.308863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51693 (local address 127.0.0.1:46279)
0808 15:56:02.308971 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:02.308975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:02.308987 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:56:02.309017 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:02.309020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:02.309082 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:02.309170 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:02.309441 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:02.309921 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:02.310740 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:02.310960 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:02.311070 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:02.311905 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:02.311909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:02.311911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:02.311930 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:02.311953 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:02.311963 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:02.312044 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:02.312132 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 15:56:02.312187 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 15:56:02.312543 28339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:02.317979 28339 catalog_manager.cc:1347] Generated new cluster ID: 0f7e111b70b642d88b6c6508cfabd930
I20260808 15:56:02.318053 28339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:02.343802 28339 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:02.344776 28339 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:02.353499 28339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a87353f4c349f7a58145b5f29be8a8: Generated new TSK 0
I20260808 15:56:02.354283 28339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:02.364099 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 28229
2026-08-08T15:56:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1761 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:02 dist-test-slave-sb2z krb5kdc[28371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:02 dist-test-slave-sb2z krb5kdc[28371](info): set up 2 sockets
Aug 08 15:56:02 dist-test-slave-sb2z krb5kdc[28371](info): commencing operation
krb5kdc: starting...
W20260808 15:56:03.573313 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:03 dist-test-slave-sb2z krb5kdc[28371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:03.696537 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:03.807301 28387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:03.807549 28387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:03.807590 28387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:03.811498 28387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:03.811563 28387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:03.811647 28387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:03.811685 28387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:03.811743 28387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:03.816754 28387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:03.817684 28387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:03.818647 28387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:03.827401 28394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:03.827437 28387 server_base.cc:1034] running on GCE node
W20260808 15:56:03.827392 28393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:03.827392 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:03.828317 28387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:03.829317 28387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:03.830506 28387 hybrid_clock.cc:648] HybridClock initialized: now 1786204563830476 us; error 44 us; skew 500 ppm
Aug 08 15:56:03 dist-test-slave-sb2z krb5kdc[28371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204563, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:03.833194 28387 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:03.834096 28387 webserver.cc:460] Webserver started at http://127.0.0.1:34095/ using document root <none> and password file <none>
I20260808 15:56:03.834641 28387 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:03.834733 28387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:03.834965 28387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:03.836983 28387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "572dc825a18f49e2b062aed7a8f90d74"
format_stamp: "Formatted at 2026-08-08 15:56:03 on dist-test-slave-sb2z"
server_key: "9a835733de593c9f34812fc2edced023"
server_key_iv: "f072aefe556e365c7fc38e55a4904b3d"
server_key_version: "encryptionkey@0"
I20260808 15:56:03.837548 28387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "572dc825a18f49e2b062aed7a8f90d74"
format_stamp: "Formatted at 2026-08-08 15:56:03 on dist-test-slave-sb2z"
server_key: "9a835733de593c9f34812fc2edced023"
server_key_iv: "f072aefe556e365c7fc38e55a4904b3d"
server_key_version: "encryptionkey@0"
I20260808 15:56:03.841259 28387 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 15:56:03.843487 28414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:03.844705 28387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:56:03.844859 28387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "572dc825a18f49e2b062aed7a8f90d74"
format_stamp: "Formatted at 2026-08-08 15:56:03 on dist-test-slave-sb2z"
server_key: "9a835733de593c9f34812fc2edced023"
server_key_iv: "f072aefe556e365c7fc38e55a4904b3d"
server_key_version: "encryptionkey@0"
I20260808 15:56:03.844970 28387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:03.849627 28387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:03.852516 28387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:03.852705 28387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:03.859798 28387 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34245
I20260808 15:56:03.859828 28488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34245 every 8 connection(s)
I20260808 15:56:03.860683 28387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:03.862711 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 28387
I20260808 15:56:03.862862 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:03.863152 31025 external_mini_cluster.cc:1428] Setting key b0a97d19f47316b51eab05e8c7e4fa09
I20260808 15:56:03.863577 28489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:03.868206 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:03.864569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56763 (local address 127.0.0.1:34245)
0808 15:56:03.865007 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:03.865021 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:03.865054 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:56:03.865681 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:03.865696 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:03.865978 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:03.866299 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:03.867048 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:03.867053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:03.867070 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:03.867085 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:03.867140 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:03.867160 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:03.867309 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:03.867380 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 15:56:03.867522 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
I20260808 15:56:03.869848 28489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74: Bootstrap starting.
I20260808 15:56:03.870844 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:03.872269 28489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:03.873095 28489 log.cc:824] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:03.874670 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:03.872906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41195 (local address 127.0.0.1:34245)
0808 15:56:03.873031 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:03.873035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:03.873225 (+ 190us) server_negotiation.cc:407] Connection header received
0808 15:56:03.873369 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:03.873372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:03.873403 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:03.873504 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:03.874262 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:03.874264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:03.874266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:03.874277 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:03.874299 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:03.874309 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:03.874377 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:03.874485 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 15:56:03.874515 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:56:03.875375 28489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74: No bootstrap required, opened a new log
I20260808 15:56:03.879036 28489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572dc825a18f49e2b062aed7a8f90d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260808 15:56:03.879251 28489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:03.879294 28489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572dc825a18f49e2b062aed7a8f90d74, State: Initialized, Role: FOLLOWER
I20260808 15:56:03.879688 28489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572dc825a18f49e2b062aed7a8f90d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260808 15:56:03.879808 28489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:03.879854 28489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:03.879923 28489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:03.880708 28489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572dc825a18f49e2b062aed7a8f90d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260808 15:56:03.881001 28489 leader_election.cc:302] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 572dc825a18f49e2b062aed7a8f90d74; no voters:
I20260808 15:56:03.881260 28489 leader_election.cc:288] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:03.881383 28502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:03.881662 28502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [term 1 LEADER]: Becoming Leader. State: Replica: 572dc825a18f49e2b062aed7a8f90d74, State: Running, Role: LEADER
I20260808 15:56:03.881987 28502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572dc825a18f49e2b062aed7a8f90d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } }
I20260808 15:56:03.882472 28489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:03.883795 28506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 572dc825a18f49e2b062aed7a8f90d74. Latest consensus state: current_term: 1 leader_uuid: "572dc825a18f49e2b062aed7a8f90d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572dc825a18f49e2b062aed7a8f90d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } } }
I20260808 15:56:03.883932 28506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:03.885453 28505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "572dc825a18f49e2b062aed7a8f90d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572dc825a18f49e2b062aed7a8f90d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34245 } } }
I20260808 15:56:03.887284 28505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74 [sys.catalog]: This master's current role is: LEADER
W20260808 15:56:03.887979 28528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:03.888062 28528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:03.888151 28517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:03.888867 28517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:03.893692 28517 catalog_manager.cc:1347] Generated new cluster ID: 6b742d7cece641b5bfee69ae841d713f
I20260808 15:56:03.893754 28517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:03.905830 28517 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:03.906147 28517 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:03.915237 28517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 572dc825a18f49e2b062aed7a8f90d74: Generated new TSK 0
I20260808 15:56:03.916069 28517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:03.920277 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 28387
2026-08-08T15:56:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:03 dist-test-slave-sb2z krb5kdc[28534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:03 dist-test-slave-sb2z krb5kdc[28534](info): set up 2 sockets
Aug 08 15:56:03 dist-test-slave-sb2z krb5kdc[28534](info): commencing operation
krb5kdc: starting...
W20260808 15:56:05.199610 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:05 dist-test-slave-sb2z krb5kdc[28534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:05.338346 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:05.456060 28552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:05.456347 28552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:05.456430 28552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:05.460441 28552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:05.460526 28552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:05.460566 28552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:05.460628 28552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:05.460667 28552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:05.465975 28552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:05.467072 28552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:05.468191 28552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:05.476688 28559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:05.476692 28560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:05.476724 28563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:05.477108 28552 server_base.cc:1034] running on GCE node
I20260808 15:56:05.477665 28552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:05.478645 28552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:05.479799 28552 hybrid_clock.cc:648] HybridClock initialized: now 1786204565479748 us; error 63 us; skew 500 ppm
Aug 08 15:56:05 dist-test-slave-sb2z krb5kdc[28534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204565, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:05.482517 28552 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:05.483304 28552 webserver.cc:460] Webserver started at http://127.0.0.1:32999/ using document root <none> and password file <none>
I20260808 15:56:05.483881 28552 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:05.483943 28552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:05.484167 28552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:05.486097 28552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "7e58429756a346c1843140eae6bbc654"
format_stamp: "Formatted at 2026-08-08 15:56:05 on dist-test-slave-sb2z"
server_key: "402f7dca7b2395eb2d576f599fdf3175"
server_key_iv: "0fd10b25244905665ec252001e802494"
server_key_version: "encryptionkey@0"
I20260808 15:56:05.486588 28552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "7e58429756a346c1843140eae6bbc654"
format_stamp: "Formatted at 2026-08-08 15:56:05 on dist-test-slave-sb2z"
server_key: "402f7dca7b2395eb2d576f599fdf3175"
server_key_iv: "0fd10b25244905665ec252001e802494"
server_key_version: "encryptionkey@0"
I20260808 15:56:05.490118 28552 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 15:56:05.492386 28575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:05.493569 28552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:05.493721 28552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "7e58429756a346c1843140eae6bbc654"
format_stamp: "Formatted at 2026-08-08 15:56:05 on dist-test-slave-sb2z"
server_key: "402f7dca7b2395eb2d576f599fdf3175"
server_key_iv: "0fd10b25244905665ec252001e802494"
server_key_version: "encryptionkey@0"
I20260808 15:56:05.493830 28552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:05.508033 28552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:05.510795 28552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:05.511011 28552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:05.518579 28552 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42593
I20260808 15:56:05.518595 28647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42593 every 8 connection(s)
I20260808 15:56:05.519582 28552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:05.522389 28648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:05.524829 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 28552
I20260808 15:56:05.525008 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:05.525297 31025 external_mini_cluster.cc:1428] Setting key 6a0557e05109bfc1077d4573b5f51b5f
I20260808 15:56:05.528384 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654: Bootstrap starting.
I20260808 15:56:05.530718 28648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:05.531468 28648 log.cc:824] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:05.533471 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654: No bootstrap required, opened a new log
Aug 08 15:56:05 dist-test-slave-sb2z krb5kdc[28534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:56:05.536850 28648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e58429756a346c1843140eae6bbc654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260808 15:56:05.537106 28648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:05.537199 28648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e58429756a346c1843140eae6bbc654, State: Initialized, Role: FOLLOWER
I20260808 15:56:05.537686 28648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e58429756a346c1843140eae6bbc654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260808 15:56:05.537842 28648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:05.537933 28648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:05.538036 28648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:05.538952 28648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e58429756a346c1843140eae6bbc654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260808 15:56:05.539305 28648 leader_election.cc:302] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e58429756a346c1843140eae6bbc654; no voters:
I20260808 15:56:05.539727 28648 leader_election.cc:288] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:05.539830 28654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:05.540058 28654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [term 1 LEADER]: Becoming Leader. State: Replica: 7e58429756a346c1843140eae6bbc654, State: Running, Role: LEADER
I20260808 15:56:05.540428 28654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e58429756a346c1843140eae6bbc654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260808 15:56:05.540834 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:05.526792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:40135 (local address 127.0.0.1:42593)
0808 15:56:05.527197 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:05.527213 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:05.527248 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:56:05.527930 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:05.527950 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:05.528287 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:05.528637 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:05.529471 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:05.530473 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:05.531423 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:05.531995 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:05.532166 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:05.534810 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:05.534815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:05.534828 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:05.534855 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:05.537265 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:05.537824 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:05.537832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:05.537839 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:05.538016 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:05.538384 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:05.538388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:05.538389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:05.538871 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:05.539163 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:05.539509 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 15:56:05.539827 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":123,"threads_started":1}
I20260808 15:56:05.541131 28648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:05.542675 28655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e58429756a346c1843140eae6bbc654. Latest consensus state: current_term: 1 leader_uuid: "7e58429756a346c1843140eae6bbc654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e58429756a346c1843140eae6bbc654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } } }
I20260808 15:56:05.542814 28655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:05.543413 28671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:05.543794 28656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e58429756a346c1843140eae6bbc654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e58429756a346c1843140eae6bbc654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } } }
I20260808 15:56:05.543895 28656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:05.544525 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:05.546259 28671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:05.553464 28671 catalog_manager.cc:1347] Generated new cluster ID: 3cab595151584b18b1ce45ba17c2e005
I20260808 15:56:05.553580 28671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:05.555860 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:05.546527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:43851 (local address 127.0.0.1:42593)
0808 15:56:05.546624 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:05.546628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:05.546946 (+ 318us) server_negotiation.cc:407] Connection header received
0808 15:56:05.547067 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:05.547070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:05.547118 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:05.547211 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:05.548076 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:05.548877 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:05.550022 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:05.550394 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:05.550505 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:05.552051 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:05.552054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:05.552057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:05.552088 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:05.553916 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:05.554488 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:05.554491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:05.554493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:05.554573 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:05.554950 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:05.554953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:05.554954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:05.555157 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:05.555294 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:05.555619 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 15:56:05.555713 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 15:56:05.570088 28671 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:05.571077 28671 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:05.576720 28671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e58429756a346c1843140eae6bbc654: Generated new TSK 0
I20260808 15:56:05.577363 28671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:05.584236 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 28552
W20260808 15:56:05.597744 28532 connection.cc:535] client connection to 127.0.0.1:42593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42593): Connection reset by peer (error 104)
2026-08-08T15:56:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1665 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:05 dist-test-slave-sb2z krb5kdc[28688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:05 dist-test-slave-sb2z krb5kdc[28688](info): set up 2 sockets
Aug 08 15:56:05 dist-test-slave-sb2z krb5kdc[28688](info): commencing operation
krb5kdc: starting...
W20260808 15:56:06.802373 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:06 dist-test-slave-sb2z krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:06.926159 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:07.038013 28706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:07.038324 28706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:07.038409 28706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:07.042397 28706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:07.042496 28706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:07.042533 28706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:07.042572 28706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:07.042603 28706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:07.047736 28706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:07.048794 28706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:07.049840 28706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:07.058315 28716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:07.058354 28715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:07.058320 28718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:07.058717 28706 server_base.cc:1034] running on GCE node
I20260808 15:56:07.059293 28706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:07.060350 28706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:07.061547 28706 hybrid_clock.cc:648] HybridClock initialized: now 1786204567061538 us; error 32 us; skew 500 ppm
Aug 08 15:56:07 dist-test-slave-sb2z krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:07.064297 28706 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:07.065173 28706 webserver.cc:460] Webserver started at http://127.0.0.1:36037/ using document root <none> and password file <none>
I20260808 15:56:07.065713 28706 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:07.065807 28706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:07.066051 28706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:07.068115 28706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "19740c73d32c488cbf6128fd4b68a172"
format_stamp: "Formatted at 2026-08-08 15:56:07 on dist-test-slave-sb2z"
server_key: "058e429dcf8efad149e17fc558e3c82b"
server_key_iv: "e88ab83373418050b523844de3bb3112"
server_key_version: "encryptionkey@0"
I20260808 15:56:07.068676 28706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "19740c73d32c488cbf6128fd4b68a172"
format_stamp: "Formatted at 2026-08-08 15:56:07 on dist-test-slave-sb2z"
server_key: "058e429dcf8efad149e17fc558e3c82b"
server_key_iv: "e88ab83373418050b523844de3bb3112"
server_key_version: "encryptionkey@0"
I20260808 15:56:07.072281 28706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:56:07.074609 28728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:07.075901 28706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:07.076052 28706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "19740c73d32c488cbf6128fd4b68a172"
format_stamp: "Formatted at 2026-08-08 15:56:07 on dist-test-slave-sb2z"
server_key: "058e429dcf8efad149e17fc558e3c82b"
server_key_iv: "e88ab83373418050b523844de3bb3112"
server_key_version: "encryptionkey@0"
I20260808 15:56:07.076169 28706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:07.095230 28706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:07.098026 28706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:07.098255 28706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:07.105368 28706 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37441
I20260808 15:56:07.105396 28803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37441 every 8 connection(s)
I20260808 15:56:07.106283 28706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:07.108932 28804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:07.112533 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 28706
I20260808 15:56:07.112704 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:07.112993 31025 external_mini_cluster.cc:1428] Setting key 2fa468b7e5a4d0fb63cb55ef72c9e201
I20260808 15:56:07.114880 28804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172: Bootstrap starting.
I20260808 15:56:07.117022 28804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:07.117786 28804 log.cc:824] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:07.119585 28804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172: No bootstrap required, opened a new log
I20260808 15:56:07.121712 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:07.114383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:40509 (local address 127.0.0.1:37441)
0808 15:56:07.114792 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:07.114806 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:07.114839 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:56:07.115496 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:07.115513 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:07.115902 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:07.116229 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:07.116625 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:07.117642 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:07.118599 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:07.119059 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:07.119254 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:07.120123 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:07.120140 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:07.120153 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:07.120501 (+ 348us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:07.120551 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:07.120574 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:07.120735 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:07.120919 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 15:56:07.121071 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260808 15:56:07.123083 28804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19740c73d32c488cbf6128fd4b68a172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37441 } }
I20260808 15:56:07.123313 28804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:07.123404 28804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19740c73d32c488cbf6128fd4b68a172, State: Initialized, Role: FOLLOWER
I20260808 15:56:07.123880 28804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19740c73d32c488cbf6128fd4b68a172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37441 } }
I20260808 15:56:07.124040 28804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:07.124140 28804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:07.124243 28804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:07.124596 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:07.125276 28804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19740c73d32c488cbf6128fd4b68a172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37441 } }
I20260808 15:56:07.125638 28804 leader_election.cc:302] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19740c73d32c488cbf6128fd4b68a172; no voters:
I20260808 15:56:07.125988 28804 leader_election.cc:288] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:07.126102 28814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:07.126386 28814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [term 1 LEADER]: Becoming Leader. State: Replica: 19740c73d32c488cbf6128fd4b68a172, State: Running, Role: LEADER
I20260808 15:56:07.126771 28814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19740c73d32c488cbf6128fd4b68a172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37441 } }
I20260808 15:56:07.127260 28804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:07.128015 28814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19740c73d32c488cbf6128fd4b68a172. Latest consensus state: current_term: 1 leader_uuid: "19740c73d32c488cbf6128fd4b68a172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19740c73d32c488cbf6128fd4b68a172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37441 } } }
I20260808 15:56:07.128147 28814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:07.128477 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19740c73d32c488cbf6128fd4b68a172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19740c73d32c488cbf6128fd4b68a172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37441 } } }
I20260808 15:56:07.128609 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:07.131735 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:07.126898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:42417 (local address 127.0.0.1:37441)
0808 15:56:07.127112 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:07.127116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:07.127185 (+ 69us) server_negotiation.cc:407] Connection header received
0808 15:56:07.127415 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:07.127418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:07.127464 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:07.127614 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:07.127940 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:07.128741 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:07.129983 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:07.130229 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:07.130348 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:07.131209 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:07.131215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:07.131219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:07.131234 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:07.131262 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:07.131275 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:07.131372 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:07.131567 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 15:56:07.131600 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
W20260808 15:56:07.132366 28843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:07.132462 28843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:07.132742 28827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:07.133490 28827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:07.138840 28827 catalog_manager.cc:1347] Generated new cluster ID: 2d890f815c594638b661a8c43c9d7d20
I20260808 15:56:07.138932 28827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:07.157811 28827 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:07.158856 28827 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:07.165670 28827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19740c73d32c488cbf6128fd4b68a172: Generated new TSK 0
I20260808 15:56:07.166316 28827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:07.170341 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 28706
W20260808 15:56:07.182559 28685 connection.cc:535] client connection to 127.0.0.1:37441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37441): Connection reset by peer (error 104)
2026-08-08T15:56:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:07 dist-test-slave-sb2z krb5kdc[28851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:07 dist-test-slave-sb2z krb5kdc[28851](info): set up 2 sockets
Aug 08 15:56:07 dist-test-slave-sb2z krb5kdc[28851](info): commencing operation
krb5kdc: starting...
W20260808 15:56:08.392222 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:08 dist-test-slave-sb2z krb5kdc[28851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:08.516866 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:08.627934 28866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:08.628180 28866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:08.628218 28866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:08.632226 28866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:08.632278 28866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:08.632297 28866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:08.632313 28866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:08.632366 28866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:08.637454 28866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:08.638423 28866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:08.639452 28866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:08.648064 28874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:08.648183 28866 server_base.cc:1034] running on GCE node
W20260808 15:56:08.648072 28875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:08.648075 28879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:08.648911 28866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:08.649924 28866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:08.651122 28866 hybrid_clock.cc:648] HybridClock initialized: now 1786204568651088 us; error 46 us; skew 500 ppm
Aug 08 15:56:08 dist-test-slave-sb2z krb5kdc[28851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204568, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:08.656657 28866 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:08.657735 28866 webserver.cc:460] Webserver started at http://127.0.0.1:36819/ using document root <none> and password file <none>
I20260808 15:56:08.658315 28866 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:08.658414 28866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:08.658658 28866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:08.660775 28866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "7746c10d3f2649bd85ecf7365a9a56fb"
format_stamp: "Formatted at 2026-08-08 15:56:08 on dist-test-slave-sb2z"
server_key: "2fc4ec537f8a6e92e339f7d2f45402a7"
server_key_iv: "a743d1af5855991f0442433608a3e1e3"
server_key_version: "encryptionkey@0"
I20260808 15:56:08.661381 28866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "7746c10d3f2649bd85ecf7365a9a56fb"
format_stamp: "Formatted at 2026-08-08 15:56:08 on dist-test-slave-sb2z"
server_key: "2fc4ec537f8a6e92e339f7d2f45402a7"
server_key_iv: "a743d1af5855991f0442433608a3e1e3"
server_key_version: "encryptionkey@0"
I20260808 15:56:08.665038 28866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:56:08.667394 28889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:08.668638 28866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:56:08.668787 28866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "7746c10d3f2649bd85ecf7365a9a56fb"
format_stamp: "Formatted at 2026-08-08 15:56:08 on dist-test-slave-sb2z"
server_key: "2fc4ec537f8a6e92e339f7d2f45402a7"
server_key_iv: "a743d1af5855991f0442433608a3e1e3"
server_key_version: "encryptionkey@0"
I20260808 15:56:08.668898 28866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:08.673437 28866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:08.676704 28866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:08.676908 28866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:08.684083 28866 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37773
I20260808 15:56:08.684095 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37773 every 8 connection(s)
I20260808 15:56:08.685002 28866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:08.687744 28965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:08.693023 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 28866
I20260808 15:56:08.693040 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb: Bootstrap starting.
I20260808 15:56:08.693179 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:08.693473 31025 external_mini_cluster.cc:1428] Setting key 05eec67955a044b8c913ddf8de7e288d
I20260808 15:56:08.695566 28965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:08.696300 28965 log.cc:824] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:08.698506 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb: No bootstrap required, opened a new log
I20260808 15:56:08.698680 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:08.694843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:54053 (local address 127.0.0.1:37773)
0808 15:56:08.695259 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:08.695274 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:08.695308 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:56:08.696019 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:08.696037 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:08.696307 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:08.696659 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:08.697461 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:08.697465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:08.697476 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:08.697485 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:08.697537 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:08.697554 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:08.697667 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:08.697777 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 15:56:08.697942 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":132,"threads_started":1}
I20260808 15:56:08.701368 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:08.702575 28965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37773 } }
I20260808 15:56:08.702821 28965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:08.702904 28965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7746c10d3f2649bd85ecf7365a9a56fb, State: Initialized, Role: FOLLOWER
I20260808 15:56:08.703370 28965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37773 } }
I20260808 15:56:08.703614 28965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:08.703712 28965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:08.703869 28965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:08.705005 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:08.703361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:59467 (local address 127.0.0.1:37773)
0808 15:56:08.703486 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:08.703490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:08.703750 (+ 260us) server_negotiation.cc:407] Connection header received
0808 15:56:08.703871 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:08.703873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:08.703901 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:08.703997 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:08.704668 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:08.704670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:08.704672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:08.704682 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:08.704705 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:08.704715 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:08.704782 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:08.704896 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 15:56:08.704926 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:56:08.705085 28965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37773 } }
I20260808 15:56:08.705426 28965 leader_election.cc:302] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7746c10d3f2649bd85ecf7365a9a56fb; no voters:
I20260808 15:56:08.705758 28965 leader_election.cc:288] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:08.705828 28980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:08.706073 28980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [term 1 LEADER]: Becoming Leader. State: Replica: 7746c10d3f2649bd85ecf7365a9a56fb, State: Running, Role: LEADER
I20260808 15:56:08.706357 28980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37773 } }
I20260808 15:56:08.707062 28965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:08.708264 28982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7746c10d3f2649bd85ecf7365a9a56fb. Latest consensus state: current_term: 1 leader_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37773 } } }
I20260808 15:56:08.708277 28983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7746c10d3f2649bd85ecf7365a9a56fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37773 } } }
I20260808 15:56:08.708463 28982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:08.708467 28983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:08.709477 28992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:08.712482 28992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:08.717854 28992 catalog_manager.cc:1347] Generated new cluster ID: 3451d8ec817a4755ae31b71fef946b30
I20260808 15:56:08.717931 28992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:08.735745 28992 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:08.736060 28992 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:08.745734 28992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7746c10d3f2649bd85ecf7365a9a56fb: Generated new TSK 0
I20260808 15:56:08.746400 28992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:08.749603 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 28866
2026-08-08T15:56:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:08 dist-test-slave-sb2z krb5kdc[29009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:08 dist-test-slave-sb2z krb5kdc[29009](info): set up 2 sockets
Aug 08 15:56:08 dist-test-slave-sb2z krb5kdc[29009](info): commencing operation
krb5kdc: starting...
W20260808 15:56:09.972970 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:10 dist-test-slave-sb2z krb5kdc[29009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:10.098093 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:10.206490 29029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:10.206768 29029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:10.206842 29029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:10.210799 29029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:10.210886 29029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:10.210927 29029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:10.210984 29029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:10.211023 29029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:10.216207 29029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:10.217216 29029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:10.218232 29029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:10.226701 29041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:10.226749 29036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:10.226701 29035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:10.226910 29029 server_base.cc:1034] running on GCE node
I20260808 15:56:10.227607 29029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:10.228673 29029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:10.229835 29029 hybrid_clock.cc:648] HybridClock initialized: now 1786204570229822 us; error 26 us; skew 500 ppm
Aug 08 15:56:10 dist-test-slave-sb2z krb5kdc[29009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204570, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:10.232669 29029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:10.233474 29029 webserver.cc:460] Webserver started at http://127.0.0.1:35319/ using document root <none> and password file <none>
I20260808 15:56:10.233974 29029 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:10.234033 29029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:10.234198 29029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:10.236219 29029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd"
format_stamp: "Formatted at 2026-08-08 15:56:10 on dist-test-slave-sb2z"
server_key: "fac0d76aba096ee2cae5a4d29bc51fc1"
server_key_iv: "dcf4068046f31bc4aa875b4ad8b05bf9"
server_key_version: "encryptionkey@0"
I20260808 15:56:10.236734 29029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd"
format_stamp: "Formatted at 2026-08-08 15:56:10 on dist-test-slave-sb2z"
server_key: "fac0d76aba096ee2cae5a4d29bc51fc1"
server_key_iv: "dcf4068046f31bc4aa875b4ad8b05bf9"
server_key_version: "encryptionkey@0"
I20260808 15:56:10.240183 29029 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:56:10.242376 29050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:10.243593 29029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:10.243729 29029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd"
format_stamp: "Formatted at 2026-08-08 15:56:10 on dist-test-slave-sb2z"
server_key: "fac0d76aba096ee2cae5a4d29bc51fc1"
server_key_iv: "dcf4068046f31bc4aa875b4ad8b05bf9"
server_key_version: "encryptionkey@0"
I20260808 15:56:10.243824 29029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:10.274832 29029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:10.277661 29029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:10.277868 29029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:10.285118 29029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38071
I20260808 15:56:10.285146 29130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38071 every 8 connection(s)
I20260808 15:56:10.286051 29029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:10.288646 29131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:10.293962 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd: Bootstrap starting.
I20260808 15:56:10.294174 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 29029
I20260808 15:56:10.294319 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:10.294612 31025 external_mini_cluster.cc:1428] Setting key d0eafd40902344c8e0cf8ef8b1ef35eb
I20260808 15:56:10.296631 29131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:10.297367 29131 log.cc:824] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:10.299175 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd: No bootstrap required, opened a new log
I20260808 15:56:10.302678 29131 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260808 15:56:10.302917 29131 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:10.303041 29131 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7a3b9ea2f942a0a409bb47c1d5dafd, State: Initialized, Role: FOLLOWER
Aug 08 15:56:10 dist-test-slave-sb2z krb5kdc[29009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:56:10.303584 29131 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260808 15:56:10.303786 29131 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:10.303862 29131 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:10.303972 29131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:10.305177 29131 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260808 15:56:10.305553 29131 leader_election.cc:302] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb7a3b9ea2f942a0a409bb47c1d5dafd; no voters:
I20260808 15:56:10.305891 29131 leader_election.cc:288] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:10.305984 29139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:10.306227 29139 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [term 1 LEADER]: Becoming Leader. State: Replica: eb7a3b9ea2f942a0a409bb47c1d5dafd, State: Running, Role: LEADER
I20260808 15:56:10.306541 29139 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260808 15:56:10.307261 29131 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:10.307716 29139 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb7a3b9ea2f942a0a409bb47c1d5dafd. Latest consensus state: current_term: 1 leader_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } } }
I20260808 15:56:10.307845 29139 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:10.308188 29140 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a3b9ea2f942a0a409bb47c1d5dafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } } }
I20260808 15:56:10.308314 29140 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:10.309921 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:10.295974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:42225 (local address 127.0.0.1:38071)
0808 15:56:10.296390 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:10.296404 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:10.296437 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:56:10.297095 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:10.297113 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:10.297443 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:10.297796 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:10.298607 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:10.299703 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:10.300553 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:10.301038 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:10.301213 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:10.304170 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:10.304179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:10.304200 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:10.304234 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:10.306193 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:10.306849 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:10.306854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:10.306860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:10.306996 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:10.307443 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:10.307447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:10.307448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:10.308144 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:10.308486 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:10.308688 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 15:56:10.308925 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
W20260808 15:56:10.312244 29156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:10.312330 29156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:10.312399 29143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:10.313160 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:10.313243 29143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:10.318696 29143 catalog_manager.cc:1347] Generated new cluster ID: 2b703b03154d41c69bcd11106c99b7bb
I20260808 15:56:10.318765 29143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:10.324785 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:10.315321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:48665 (local address 127.0.0.1:38071)
0808 15:56:10.315421 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:10.315426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:10.315801 (+ 375us) server_negotiation.cc:407] Connection header received
0808 15:56:10.315916 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:10.315919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:10.315971 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:10.316056 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:10.316894 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:10.318113 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:10.319311 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:10.319668 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:10.319759 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:10.321383 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:10.321386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:10.321389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:10.321412 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:10.322980 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:10.323512 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:10.323515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:10.323517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:10.323619 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:10.323967 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:10.323969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:10.323971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:10.324150 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:10.324287 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:10.324551 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 15:56:10.324646 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 15:56:10.332219 29143 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:10.333341 29143 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:10.342303 29143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb7a3b9ea2f942a0a409bb47c1d5dafd: Generated new TSK 0
I20260808 15:56:10.342938 29143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:10.350325 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 29029
W20260808 15:56:10.362290 29006 connection.cc:535] client connection to 127.0.0.1:38071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38071): Connection reset by peer (error 104)
2026-08-08T15:56:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:10 dist-test-slave-sb2z krb5kdc[29171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:10 dist-test-slave-sb2z krb5kdc[29171](info): set up 2 sockets
Aug 08 15:56:10 dist-test-slave-sb2z krb5kdc[29171](info): commencing operation
krb5kdc: starting...
W20260808 15:56:11.582163 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:11 dist-test-slave-sb2z krb5kdc[29171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:11.712631 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:11.825379 29187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:11.825708 29187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:11.825788 29187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:11.829818 29187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:11.829906 29187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:11.829959 29187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:11.830018 29187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:11.830061 29187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:11.835265 29187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:11.836337 29187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:11.837391 29187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:11.845768 29205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:11.845685 29200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:11.845683 29201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:11.846036 29187 server_base.cc:1034] running on GCE node
I20260808 15:56:11.846707 29187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:11.847831 29187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:11.849021 29187 hybrid_clock.cc:648] HybridClock initialized: now 1786204571849000 us; error 32 us; skew 500 ppm
Aug 08 15:56:11 dist-test-slave-sb2z krb5kdc[29171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204571, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:11.851814 29187 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:11.852646 29187 webserver.cc:460] Webserver started at http://127.0.0.1:44459/ using document root <none> and password file <none>
I20260808 15:56:11.853199 29187 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:11.853293 29187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:11.853525 29187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:11.855597 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "51083eb800fa4bb39a401ef0558ae858"
format_stamp: "Formatted at 2026-08-08 15:56:11 on dist-test-slave-sb2z"
server_key: "0e5a94770534b310aededefb8bbdcab8"
server_key_iv: "785e7f3da3bb9ef6c5da62e5967eb4ba"
server_key_version: "encryptionkey@0"
I20260808 15:56:11.856178 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "51083eb800fa4bb39a401ef0558ae858"
format_stamp: "Formatted at 2026-08-08 15:56:11 on dist-test-slave-sb2z"
server_key: "0e5a94770534b310aededefb8bbdcab8"
server_key_iv: "785e7f3da3bb9ef6c5da62e5967eb4ba"
server_key_version: "encryptionkey@0"
I20260808 15:56:11.859833 29187 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:56:11.862078 29220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:11.863225 29187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:56:11.863361 29187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "51083eb800fa4bb39a401ef0558ae858"
format_stamp: "Formatted at 2026-08-08 15:56:11 on dist-test-slave-sb2z"
server_key: "0e5a94770534b310aededefb8bbdcab8"
server_key_iv: "785e7f3da3bb9ef6c5da62e5967eb4ba"
server_key_version: "encryptionkey@0"
I20260808 15:56:11.863468 29187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:11.878499 29187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:11.881675 29187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:11.881872 29187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:11.888680 29187 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36977
I20260808 15:56:11.888693 29291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36977 every 8 connection(s)
I20260808 15:56:11.889595 29187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:11.892308 29292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:11.897589 29292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858: Bootstrap starting.
I20260808 15:56:11.899127 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 29187
I20260808 15:56:11.899279 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:11.899654 31025 external_mini_cluster.cc:1428] Setting key 2470be5d2f1e993a84f4f4d1a197e092
I20260808 15:56:11.900292 29292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:11.901154 29292 log.cc:824] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:11.903353 29292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858: No bootstrap required, opened a new log
I20260808 15:56:11.907471 29292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51083eb800fa4bb39a401ef0558ae858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 15:56:11.907748 29292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:11.907814 29292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51083eb800fa4bb39a401ef0558ae858, State: Initialized, Role: FOLLOWER
I20260808 15:56:11.908308 29292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51083eb800fa4bb39a401ef0558ae858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 15:56:11.908465 29292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:11.908532 29292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:11.908625 29292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:11.909394 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:11.900784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:60979 (local address 127.0.0.1:36977)
0808 15:56:11.901270 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:11.901290 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:11.901329 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:56:11.902091 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:11.902113 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:11.902511 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:11.902868 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:11.903267 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:11.904735 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:11.905590 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:11.906393 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:11.906556 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:11.907383 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:11.907390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:11.907405 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:11.907932 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:11.908005 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:11.908038 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:11.908164 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:11.908321 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 15:56:11.908449 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":159,"threads_started":1}
I20260808 15:56:11.909705 29292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51083eb800fa4bb39a401ef0558ae858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 15:56:11.910043 29292 leader_election.cc:302] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51083eb800fa4bb39a401ef0558ae858; no voters:
I20260808 15:56:11.910383 29292 leader_election.cc:288] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:11.910483 29299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:11.910681 29299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [term 1 LEADER]: Becoming Leader. State: Replica: 51083eb800fa4bb39a401ef0558ae858, State: Running, Role: LEADER
I20260808 15:56:11.910959 29299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51083eb800fa4bb39a401ef0558ae858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 15:56:11.911628 29292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:11.912199 29301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51083eb800fa4bb39a401ef0558ae858. Latest consensus state: current_term: 1 leader_uuid: "51083eb800fa4bb39a401ef0558ae858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51083eb800fa4bb39a401ef0558ae858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } } }
I20260808 15:56:11.912379 29301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:11.912930 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51083eb800fa4bb39a401ef0558ae858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51083eb800fa4bb39a401ef0558ae858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } } }
I20260808 15:56:11.913031 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:11.913365 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:11.914302 29306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:11.918167 29306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:11.923759 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:11.915919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:52425 (local address 127.0.0.1:36977)
0808 15:56:11.916017 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:11.916021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:11.919807 (+ 3786us) server_negotiation.cc:407] Connection header received
0808 15:56:11.919957 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:11.919973 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:11.920022 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:11.920129 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:11.920402 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:11.920934 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:11.921752 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:11.921994 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:11.922107 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:11.923171 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:11.923176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:11.923179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:11.923192 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:11.923220 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:11.923234 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:11.923334 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:11.923576 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 15:56:11.923622 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 15:56:11.925300 29306 catalog_manager.cc:1347] Generated new cluster ID: 6699a948a0e84386b5cdb1f81eab4394
I20260808 15:56:11.925369 29306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:11.940564 29306 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:11.941730 29306 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:11.949276 29306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51083eb800fa4bb39a401ef0558ae858: Generated new TSK 0
I20260808 15:56:11.950040 29306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:11.958904 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 29187
W20260808 15:56:11.976622 29167 connection.cc:535] client connection to 127.0.0.1:36977 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36977): Connection reset by peer (error 104)
2026-08-08T15:56:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:12 dist-test-slave-sb2z krb5kdc[29337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:12 dist-test-slave-sb2z krb5kdc[29337](info): set up 2 sockets
Aug 08 15:56:12 dist-test-slave-sb2z krb5kdc[29337](info): commencing operation
krb5kdc: starting...
W20260808 15:56:13.169992 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:13 dist-test-slave-sb2z krb5kdc[29337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:56:13.296121 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42169
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:13.408764 29358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:13.409016 29358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:13.409055 29358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:13.412938 29358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:13.412992 29358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:13.413010 29358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:13.413026 29358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:13.413089 29358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:13.418118 29358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42169
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:13.419070 29358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:13.420055 29358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:13.428092 29371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:13.428107 29366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:13.428141 29367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:13.428391 29358 server_base.cc:1034] running on GCE node
I20260808 15:56:13.429013 29358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:13.430006 29358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:13.431170 29358 hybrid_clock.cc:648] HybridClock initialized: now 1786204573431153 us; error 30 us; skew 500 ppm
Aug 08 15:56:13 dist-test-slave-sb2z krb5kdc[29337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204573, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:13.433911 29358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:56:13.434693 29358 webserver.cc:460] Webserver started at http://127.0.0.1:42641/ using document root <none> and password file <none>
I20260808 15:56:13.435185 29358 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:13.435242 29358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:13.435397 29358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:13.437350 29358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "8604b76a19fb42a29421a71e30bfcb78"
format_stamp: "Formatted at 2026-08-08 15:56:13 on dist-test-slave-sb2z"
server_key: "2164c4f76b79b7a1b33d3e77484d07a5"
server_key_iv: "64edf5e06649d523333d40ae217a04ee"
server_key_version: "encryptionkey@0"
I20260808 15:56:13.437847 29358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "8604b76a19fb42a29421a71e30bfcb78"
format_stamp: "Formatted at 2026-08-08 15:56:13 on dist-test-slave-sb2z"
server_key: "2164c4f76b79b7a1b33d3e77484d07a5"
server_key_iv: "64edf5e06649d523333d40ae217a04ee"
server_key_version: "encryptionkey@0"
I20260808 15:56:13.441322 29358 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:56:13.443488 29380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:13.444752 29358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:56:13.444852 29358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "8604b76a19fb42a29421a71e30bfcb78"
format_stamp: "Formatted at 2026-08-08 15:56:13 on dist-test-slave-sb2z"
server_key: "2164c4f76b79b7a1b33d3e77484d07a5"
server_key_iv: "64edf5e06649d523333d40ae217a04ee"
server_key_version: "encryptionkey@0"
I20260808 15:56:13.444931 29358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:13.449195 29358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:13.452298 29358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:13.452435 29358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:13.461107 29358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42169
I20260808 15:56:13.461148 29460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42169 every 8 connection(s)
I20260808 15:56:13.462015 29358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:13.462276 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 29358
I20260808 15:56:13.462425 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:13.462786 31025 external_mini_cluster.cc:1428] Setting key 0b4eeedd41539d8b9917145d62672d8f
I20260808 15:56:13.465279 29461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:13.467919 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:13.464221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:42517 (local address 127.0.0.1:42169)
0808 15:56:13.464642 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:13.464657 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:13.464690 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:56:13.465301 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:13.465316 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:13.465648 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:13.465980 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:13.466720 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:13.466726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:13.466748 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:13.466758 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:13.466806 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:13.466824 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:13.466974 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:13.467045 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 15:56:13.467207 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":133,"threads_started":1}
I20260808 15:56:13.470739 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:13.471443 29461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78: Bootstrap starting.
I20260808 15:56:13.473955 29461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:13.474390 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:13.472787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:48771 (local address 127.0.0.1:42169)
0808 15:56:13.472914 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:13.472918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:13.473120 (+ 202us) server_negotiation.cc:407] Connection header received
0808 15:56:13.473229 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:13.473231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:13.473257 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:13.473320 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:13.473963 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:13.473966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:13.473968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:13.473977 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:13.474000 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:13.474033 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:13.474153 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:13.474220 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 15:56:13.474260 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:56:13.474844 29461 log.cc:824] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:13.476651 29461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78: No bootstrap required, opened a new log
I20260808 15:56:13.480156 29461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8604b76a19fb42a29421a71e30bfcb78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42169 } }
I20260808 15:56:13.480392 29461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:13.480469 29461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8604b76a19fb42a29421a71e30bfcb78, State: Initialized, Role: FOLLOWER
I20260808 15:56:13.480877 29461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8604b76a19fb42a29421a71e30bfcb78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42169 } }
I20260808 15:56:13.481032 29461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:13.481101 29461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:13.481204 29461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:13.482072 29461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8604b76a19fb42a29421a71e30bfcb78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42169 } }
I20260808 15:56:13.482417 29461 leader_election.cc:302] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8604b76a19fb42a29421a71e30bfcb78; no voters:
I20260808 15:56:13.482757 29461 leader_election.cc:288] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:13.482867 29472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:13.483137 29472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [term 1 LEADER]: Becoming Leader. State: Replica: 8604b76a19fb42a29421a71e30bfcb78, State: Running, Role: LEADER
I20260808 15:56:13.483476 29472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8604b76a19fb42a29421a71e30bfcb78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42169 } }
I20260808 15:56:13.484038 29461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:13.485183 29475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8604b76a19fb42a29421a71e30bfcb78. Latest consensus state: current_term: 1 leader_uuid: "8604b76a19fb42a29421a71e30bfcb78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8604b76a19fb42a29421a71e30bfcb78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42169 } } }
I20260808 15:56:13.485316 29475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:13.485687 29474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8604b76a19fb42a29421a71e30bfcb78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8604b76a19fb42a29421a71e30bfcb78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42169 } } }
I20260808 15:56:13.485798 29474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78 [sys.catalog]: This master's current role is: LEADER
W20260808 15:56:13.488899 29496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:13.488968 29496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:13.489048 29484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:13.489741 29484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:13.495203 29484 catalog_manager.cc:1347] Generated new cluster ID: 1ed57508ffcf4beea3c496bf8e043c67
I20260808 15:56:13.495275 29484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:13.511878 29484 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:13.512220 29484 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:13.518077 29484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8604b76a19fb42a29421a71e30bfcb78: Generated new TSK 0
I20260808 15:56:13.518725 29484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:13.526021 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 29358
2026-08-08T15:56:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:13 dist-test-slave-sb2z krb5kdc[29504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:13 dist-test-slave-sb2z krb5kdc[29504](info): set up 2 sockets
Aug 08 15:56:13 dist-test-slave-sb2z krb5kdc[29504](info): commencing operation
krb5kdc: starting...
W20260808 15:56:14.751984 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:14 dist-test-slave-sb2z krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:14.875415 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45265
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:14.992273 29520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:14.992532 29520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:14.992576 29520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:14.996688 29520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:14.996745 29520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:14.996765 29520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:14.996781 29520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:14.996796 29520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:15.001988 29520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:15.002926 29520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:15.003990 29520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:15.012730 29532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:15.012717 29528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:15.012739 29529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:15.012948 29520 server_base.cc:1034] running on GCE node
I20260808 15:56:15.013633 29520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:15.014657 29520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:15.015816 29520 hybrid_clock.cc:648] HybridClock initialized: now 1786204575015787 us; error 41 us; skew 500 ppm
Aug 08 15:56:15 dist-test-slave-sb2z krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204575, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:15.022138 29520 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:15.023044 29520 webserver.cc:460] Webserver started at http://127.30.76.126:40283/ using document root <none> and password file <none>
I20260808 15:56:15.023679 29520 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:15.023777 29520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:15.024013 29520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:15.026074 29520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "dce23656d651490bb64abca6a9841380"
format_stamp: "Formatted at 2026-08-08 15:56:15 on dist-test-slave-sb2z"
server_key: "5f62a3b7c0e2b60e842a7021cee0c55b"
server_key_iv: "8748fac35df3e71c74ecc9de28f7e578"
server_key_version: "encryptionkey@0"
I20260808 15:56:15.026643 29520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "dce23656d651490bb64abca6a9841380"
format_stamp: "Formatted at 2026-08-08 15:56:15 on dist-test-slave-sb2z"
server_key: "5f62a3b7c0e2b60e842a7021cee0c55b"
server_key_iv: "8748fac35df3e71c74ecc9de28f7e578"
server_key_version: "encryptionkey@0"
I20260808 15:56:15.030331 29520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 15:56:15.033182 29543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:15.034402 29520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:56:15.034550 29520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "dce23656d651490bb64abca6a9841380"
format_stamp: "Formatted at 2026-08-08 15:56:15 on dist-test-slave-sb2z"
server_key: "5f62a3b7c0e2b60e842a7021cee0c55b"
server_key_iv: "8748fac35df3e71c74ecc9de28f7e578"
server_key_version: "encryptionkey@0"
I20260808 15:56:15.034686 29520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:15.050114 29520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:15.052982 29520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:15.053201 29520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:15.060461 29520 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45265
I20260808 15:56:15.060478 29622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45265 every 8 connection(s)
I20260808 15:56:15.061510 29520 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:15.061728 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 29520
I20260808 15:56:15.061872 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:15.062256 31025 external_mini_cluster.cc:1428] Setting key 7548899deac89c24ae005a0be4caef71
I20260808 15:56:15.065093 29624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:15.070323 29624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380: Bootstrap starting.
Aug 08 15:56:15 dist-test-slave-sb2z krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:56:15.072804 29624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:15.073518 29624 log.cc:824] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:15.075332 29624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380: No bootstrap required, opened a new log
I20260808 15:56:15.076278 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:15.063494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49375 (local address 127.30.76.126:45265)
0808 15:56:15.063976 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:15.063992 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:15.064026 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:56:15.064669 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:15.064684 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:15.065005 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:15.065320 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:15.066369 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:15.067367 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:15.068314 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:15.068776 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:15.068947 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:15.071504 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:15.071509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:15.071521 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:15.071592 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:15.073370 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:15.073934 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:15.073944 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:15.073948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:15.074057 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:15.074411 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:15.074419 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:15.074420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:15.074792 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:15.075024 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:15.075325 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 15:56:15.075587 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":146,"threads_started":1}
I20260808 15:56:15.078274 29624 raft_consensus.cc:348] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce23656d651490bb64abca6a9841380" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45265 } }
I20260808 15:56:15.078553 29624 raft_consensus.cc:374] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:15.078676 29624 raft_consensus.cc:729] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce23656d651490bb64abca6a9841380, State: Initialized, Role: FOLLOWER
I20260808 15:56:15.079150 29624 consensus_queue.cc:260] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce23656d651490bb64abca6a9841380" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45265 } }
I20260808 15:56:15.079304 29624 raft_consensus.cc:388] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:15.079373 29624 raft_consensus.cc:482] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:15.079474 29624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:15.079802 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:15.080385 29624 raft_consensus.cc:504] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce23656d651490bb64abca6a9841380" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45265 } }
I20260808 15:56:15.080717 29624 leader_election.cc:302] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dce23656d651490bb64abca6a9841380; no voters:
I20260808 15:56:15.081048 29624 leader_election.cc:288] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:15.081148 29634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:15.081483 29634 raft_consensus.cc:686] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [term 1 LEADER]: Becoming Leader. State: Replica: dce23656d651490bb64abca6a9841380, State: Running, Role: LEADER
I20260808 15:56:15.081836 29634 consensus_queue.cc:237] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce23656d651490bb64abca6a9841380" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45265 } }
I20260808 15:56:15.082358 29624 sys_catalog.cc:564] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:15.082978 29638 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dce23656d651490bb64abca6a9841380. Latest consensus state: current_term: 1 leader_uuid: "dce23656d651490bb64abca6a9841380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce23656d651490bb64abca6a9841380" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45265 } } }
I20260808 15:56:15.083110 29638 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:15.083918 29637 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dce23656d651490bb64abca6a9841380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce23656d651490bb64abca6a9841380" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45265 } } }
I20260808 15:56:15.084002 29637 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:15.084861 29646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:15.087698 29646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:15.092636 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:15.081824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33119 (local address 127.30.76.126:45265)
0808 15:56:15.081922 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:15.081927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:15.083787 (+ 1860us) server_negotiation.cc:407] Connection header received
0808 15:56:15.083982 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:15.083988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:15.084040 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:15.084101 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:15.085138 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:15.085677 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:15.086741 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:15.087067 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:15.087188 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:15.088834 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:15.088840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:15.088846 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:15.088871 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:15.090538 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:15.091126 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:15.091129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:15.091131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:15.091232 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:15.091654 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:15.091657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:15.091659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:15.091871 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:15.092043 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:15.092374 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 15:56:15.092478 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 15:56:15.093488 29646 catalog_manager.cc:1347] Generated new cluster ID: f15b46568bd54b319ad363385a3f5cd8
I20260808 15:56:15.093544 29646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:15.118834 29646 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:15.120118 29646 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:15.128659 29646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dce23656d651490bb64abca6a9841380: Generated new TSK 0
I20260808 15:56:15.129420 29646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:15.134807 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 29520
W20260808 15:56:15.149621 29501 connection.cc:535] client connection to 127.30.76.126:45265 recv error: Network error: failed to read from TLS socket (remote: 127.30.76.126:45265): Connection reset by peer (error 104)
2026-08-08T15:56:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:15 dist-test-slave-sb2z krb5kdc[29668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:15 dist-test-slave-sb2z krb5kdc[29668](info): set up 2 sockets
Aug 08 15:56:15 dist-test-slave-sb2z krb5kdc[29668](info): commencing operation
krb5kdc: starting...
W20260808 15:56:16.349808 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:16 dist-test-slave-sb2z krb5kdc[29668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:16.473738 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39329
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:16.585038 29683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:16.585289 29683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:16.585328 29683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:16.589211 29683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:16.589263 29683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:16.589282 29683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:16.589298 29683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:16.589313 29683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:16.594307 29683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:16.595204 29683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:16.596208 29683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:16.604722 29693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:16.604838 29683 server_base.cc:1034] running on GCE node
W20260808 15:56:16.604789 29696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:16.604722 29698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:16.605563 29683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:16.606549 29683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:16.607694 29683 hybrid_clock.cc:648] HybridClock initialized: now 1786204576607675 us; error 30 us; skew 500 ppm
Aug 08 15:56:16 dist-test-slave-sb2z krb5kdc[29668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204576, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:16.610394 29683 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:16.611179 29683 webserver.cc:460] Webserver started at http://127.30.76.126:40731/ using document root <none> and password file <none>
I20260808 15:56:16.611742 29683 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:16.611806 29683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:16.611966 29683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:16.613902 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "2aedeb2e257c41ea9e02cac0c32df330"
format_stamp: "Formatted at 2026-08-08 15:56:16 on dist-test-slave-sb2z"
server_key: "b626ad3bfe9e63c6bfdfeb4add1455c6"
server_key_iv: "4b40c2bd6ed0353ad7b8718dbd2c9c1a"
server_key_version: "encryptionkey@0"
I20260808 15:56:16.614411 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "2aedeb2e257c41ea9e02cac0c32df330"
format_stamp: "Formatted at 2026-08-08 15:56:16 on dist-test-slave-sb2z"
server_key: "b626ad3bfe9e63c6bfdfeb4add1455c6"
server_key_iv: "4b40c2bd6ed0353ad7b8718dbd2c9c1a"
server_key_version: "encryptionkey@0"
I20260808 15:56:16.617921 29683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:56:16.620198 29707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:16.621392 29683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:56:16.621493 29683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "2aedeb2e257c41ea9e02cac0c32df330"
format_stamp: "Formatted at 2026-08-08 15:56:16 on dist-test-slave-sb2z"
server_key: "b626ad3bfe9e63c6bfdfeb4add1455c6"
server_key_iv: "4b40c2bd6ed0353ad7b8718dbd2c9c1a"
server_key_version: "encryptionkey@0"
I20260808 15:56:16.621575 29683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:16.647830 29683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:16.650517 29683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:16.650667 29683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:16.657405 29683 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39329
I20260808 15:56:16.657419 29778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39329 every 8 connection(s)
I20260808 15:56:16.658337 29683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:16.660320 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 29683
I20260808 15:56:16.660475 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:16.660784 31025 external_mini_cluster.cc:1428] Setting key 9c0c8711d4b449ec95f5c160f73e7fec
I20260808 15:56:16.661418 29780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:16.667163 29780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330: Bootstrap starting.
I20260808 15:56:16.669723 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:16.662103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32881 (local address 127.30.76.126:39329)
0808 15:56:16.662563 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:16.662577 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:16.662610 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:56:16.663286 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:16.663303 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:16.663684 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:16.664000 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:16.664384 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:16.665322 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:16.666246 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:16.666710 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:16.666875 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:16.667842 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:16.667848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:16.667862 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:16.668330 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:16.668401 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:16.668436 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:16.668586 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:16.668723 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 15:56:16.668846 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":179,"threads_started":1}
I20260808 15:56:16.669926 29780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:16.670773 29780 log.cc:824] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:16.672791 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:16.672963 29780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330: No bootstrap required, opened a new log
I20260808 15:56:16.676889 29780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aedeb2e257c41ea9e02cac0c32df330" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39329 } }
I20260808 15:56:16.677137 29780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:16.677217 29780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aedeb2e257c41ea9e02cac0c32df330, State: Initialized, Role: FOLLOWER
I20260808 15:56:16.677661 29780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aedeb2e257c41ea9e02cac0c32df330" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39329 } }
I20260808 15:56:16.677812 29780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:16.677899 29780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:16.677997 29780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:16.679097 29780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aedeb2e257c41ea9e02cac0c32df330" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39329 } }
I20260808 15:56:16.679473 29780 leader_election.cc:302] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aedeb2e257c41ea9e02cac0c32df330; no voters:
I20260808 15:56:16.679680 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:16.674741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35935 (local address 127.30.76.126:39329)
0808 15:56:16.674847 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:16.674850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:16.675786 (+ 936us) server_negotiation.cc:407] Connection header received
0808 15:56:16.675902 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:16.675904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:16.675945 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:16.676025 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:16.676305 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:16.676808 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:16.677742 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:16.678106 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:16.678211 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:16.679086 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:16.679089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:16.679092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:16.679105 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:16.679142 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:16.679153 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:16.679251 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:16.679404 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 15:56:16.679442 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 15:56:16.679894 29780 leader_election.cc:288] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:16.679975 29794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:16.680187 29794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [term 1 LEADER]: Becoming Leader. State: Replica: 2aedeb2e257c41ea9e02cac0c32df330, State: Running, Role: LEADER
I20260808 15:56:16.680534 29794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aedeb2e257c41ea9e02cac0c32df330" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39329 } }
I20260808 15:56:16.681198 29780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:16.682307 29796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aedeb2e257c41ea9e02cac0c32df330. Latest consensus state: current_term: 1 leader_uuid: "2aedeb2e257c41ea9e02cac0c32df330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aedeb2e257c41ea9e02cac0c32df330" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39329 } } }
I20260808 15:56:16.682650 29796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:16.682548 29795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aedeb2e257c41ea9e02cac0c32df330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aedeb2e257c41ea9e02cac0c32df330" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39329 } } }
I20260808 15:56:16.682962 29795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330 [sys.catalog]: This master's current role is: LEADER
W20260808 15:56:16.686211 29816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:16.686276 29816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:16.686358 29810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:16.687310 29810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:16.692662 29810 catalog_manager.cc:1347] Generated new cluster ID: 0fd60785179748258123802c2d0e7ea7
I20260808 15:56:16.692723 29810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:16.703565 29810 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:16.704417 29810 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:16.709551 29810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aedeb2e257c41ea9e02cac0c32df330: Generated new TSK 0
I20260808 15:56:16.710249 29810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:16.721020 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 29683
W20260808 15:56:16.732281 29666 connection.cc:535] client connection to 127.30.76.126:39329 recv error: Network error: failed to read from TLS socket (remote: 127.30.76.126:39329): Connection reset by peer (error 104)
2026-08-08T15:56:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:16 dist-test-slave-sb2z krb5kdc[29822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:16 dist-test-slave-sb2z krb5kdc[29822](info): set up 2 sockets
Aug 08 15:56:16 dist-test-slave-sb2z krb5kdc[29822](info): commencing operation
krb5kdc: starting...
W20260808 15:56:17.925693 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:18 dist-test-slave-sb2z krb5kdc[29822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:18.051911 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39649
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:18.163049 29845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:18.163288 29845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:18.163327 29845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:18.167256 29845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:18.167322 29845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:18.167342 29845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:18.167377 29845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:18.167392 29845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:18.172683 29845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:18.173578 29845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:18.174517 29845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:18.182780 29854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:18.182821 29857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:18.182871 29860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:18.183044 29845 server_base.cc:1034] running on GCE node
I20260808 15:56:18.183686 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:18.184737 29845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:18.185896 29845 hybrid_clock.cc:648] HybridClock initialized: now 1786204578185868 us; error 41 us; skew 500 ppm
Aug 08 15:56:18 dist-test-slave-sb2z krb5kdc[29822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204578, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:18.188593 29845 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:18.189466 29845 webserver.cc:460] Webserver started at http://127.30.76.126:34363/ using document root <none> and password file <none>
I20260808 15:56:18.189989 29845 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:18.190048 29845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:18.190300 29845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:18.192291 29845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb"
format_stamp: "Formatted at 2026-08-08 15:56:18 on dist-test-slave-sb2z"
server_key: "a2dde72c1651aeab4734de257da0f44d"
server_key_iv: "52f24d3fd67617ab7978a7967217aa10"
server_key_version: "encryptionkey@0"
I20260808 15:56:18.192823 29845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb"
format_stamp: "Formatted at 2026-08-08 15:56:18 on dist-test-slave-sb2z"
server_key: "a2dde72c1651aeab4734de257da0f44d"
server_key_iv: "52f24d3fd67617ab7978a7967217aa10"
server_key_version: "encryptionkey@0"
I20260808 15:56:18.196290 29845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:56:18.198487 29871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:18.199698 29845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:18.199816 29845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb"
format_stamp: "Formatted at 2026-08-08 15:56:18 on dist-test-slave-sb2z"
server_key: "a2dde72c1651aeab4734de257da0f44d"
server_key_iv: "52f24d3fd67617ab7978a7967217aa10"
server_key_version: "encryptionkey@0"
I20260808 15:56:18.199909 29845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:18.204278 29845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:18.206821 29845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:18.207001 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:18.214021 29845 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39649
I20260808 15:56:18.214051 29955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39649 every 8 connection(s)
I20260808 15:56:18.214870 29845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:18.217490 29956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:18.217921 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 29845
I20260808 15:56:18.218075 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:18.218377 31025 external_mini_cluster.cc:1428] Setting key 88f7cd063c7b84816d1ef40f578ade67
I20260808 15:56:18.223412 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:18.219793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35983 (local address 127.30.76.126:39649)
0808 15:56:18.220212 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:18.220226 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:18.220260 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:56:18.220910 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:18.220941 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:18.221210 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:18.221534 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:18.222318 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:18.222322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:18.222334 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:18.222345 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:18.222398 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:18.222416 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:18.222559 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:18.222633 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 15:56:18.222783 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":126,"threads_started":1}
I20260808 15:56:18.224268 29956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb: Bootstrap starting.
I20260808 15:56:18.226061 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:18.226660 29956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:18.227463 29956 log.cc:824] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:18.229681 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:18.228117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43595 (local address 127.30.76.126:39649)
0808 15:56:18.228254 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:18.228259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:18.228443 (+ 184us) server_negotiation.cc:407] Connection header received
0808 15:56:18.228559 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:18.228561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:18.228587 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:18.228655 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:18.229294 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:18.229297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:18.229298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:18.229308 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:18.229331 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:18.229341 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:18.229406 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:18.229515 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 15:56:18.229543 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:56:18.229748 29956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb: No bootstrap required, opened a new log
I20260808 15:56:18.233314 29956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39649 } }
I20260808 15:56:18.233558 29956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:18.233651 29956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cdf7eaa17f24a3997baf0271eaa1bdb, State: Initialized, Role: FOLLOWER
I20260808 15:56:18.234042 29956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39649 } }
I20260808 15:56:18.234165 29956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:18.234212 29956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:18.234279 29956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:18.235224 29956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39649 } }
I20260808 15:56:18.235610 29956 leader_election.cc:302] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cdf7eaa17f24a3997baf0271eaa1bdb; no voters:
I20260808 15:56:18.235908 29956 leader_election.cc:288] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:18.236035 29971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:18.236287 29971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [term 1 LEADER]: Becoming Leader. State: Replica: 3cdf7eaa17f24a3997baf0271eaa1bdb, State: Running, Role: LEADER
I20260808 15:56:18.236619 29971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39649 } }
I20260808 15:56:18.237116 29956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:18.238281 29974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cdf7eaa17f24a3997baf0271eaa1bdb. Latest consensus state: current_term: 1 leader_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39649 } } }
I20260808 15:56:18.238421 29974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:18.238682 29973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cdf7eaa17f24a3997baf0271eaa1bdb" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39649 } } }
I20260808 15:56:18.238792 29982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:18.239449 29973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:18.242121 29982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:18.247032 29982 catalog_manager.cc:1347] Generated new cluster ID: ad512f94f6c54adcba69444623d958fe
I20260808 15:56:18.247094 29982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:18.260274 29982 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:18.260572 29982 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:18.265002 29982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cdf7eaa17f24a3997baf0271eaa1bdb: Generated new TSK 0
I20260808 15:56:18.265569 29982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:18.269850 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 29845
2026-08-08T15:56:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:18 dist-test-slave-sb2z krb5kdc[30009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:18 dist-test-slave-sb2z krb5kdc[30009](info): set up 2 sockets
Aug 08 15:56:18 dist-test-slave-sb2z krb5kdc[30009](info): commencing operation
krb5kdc: starting...
W20260808 15:56:19.500758 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:19 dist-test-slave-sb2z krb5kdc[30009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:19.626223 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39603
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:19.737083 30025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:19.737375 30025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:19.737452 30025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:19.741397 30025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:19.741492 30025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:19.741530 30025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:19.741590 30025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:19.741639 30025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:19.746709 30025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:19.747761 30025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:19.748751 30025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:19.757462 30034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:19.757505 30031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:19.757773 30025 server_base.cc:1034] running on GCE node
W20260808 15:56:19.757464 30036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:19.758534 30025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:19.759693 30025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:19.760874 30025 hybrid_clock.cc:648] HybridClock initialized: now 1786204579760857 us; error 30 us; skew 500 ppm
Aug 08 15:56:19 dist-test-slave-sb2z krb5kdc[30009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204579, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:19.763623 30025 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:19.764541 30025 webserver.cc:460] Webserver started at http://127.30.76.126:36333/ using document root <none> and password file <none>
I20260808 15:56:19.765105 30025 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:19.765204 30025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:19.765443 30025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:19.767452 30025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "f2e67e3f582a46fdb3b9525c477bd9e9"
format_stamp: "Formatted at 2026-08-08 15:56:19 on dist-test-slave-sb2z"
server_key: "ceefb18f6b810c28c20f63f52479163d"
server_key_iv: "1850836f3dc75cebcebca9a98a854f20"
server_key_version: "encryptionkey@0"
I20260808 15:56:19.768095 30025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "f2e67e3f582a46fdb3b9525c477bd9e9"
format_stamp: "Formatted at 2026-08-08 15:56:19 on dist-test-slave-sb2z"
server_key: "ceefb18f6b810c28c20f63f52479163d"
server_key_iv: "1850836f3dc75cebcebca9a98a854f20"
server_key_version: "encryptionkey@0"
I20260808 15:56:19.771905 30025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:56:19.774257 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:19.775395 30025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:56:19.775614 30025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "f2e67e3f582a46fdb3b9525c477bd9e9"
format_stamp: "Formatted at 2026-08-08 15:56:19 on dist-test-slave-sb2z"
server_key: "ceefb18f6b810c28c20f63f52479163d"
server_key_iv: "1850836f3dc75cebcebca9a98a854f20"
server_key_version: "encryptionkey@0"
I20260808 15:56:19.775734 30025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:19.790618 30025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:19.793356 30025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:19.793555 30025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:19.800745 30025 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39603
I20260808 15:56:19.800773 30121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39603 every 8 connection(s)
I20260808 15:56:19.801663 30025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:19.802415 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 30025
I20260808 15:56:19.802567 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:19.802878 31025 external_mini_cluster.cc:1428] Setting key e4c59ba541ab2602e82549df0e533c17
I20260808 15:56:19.804925 30122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:19.810416 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9: Bootstrap starting.
Aug 08 15:56:19 dist-test-slave-sb2z krb5kdc[30009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:56:19.813052 30122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:19.813861 30122 log.cc:824] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:19.815804 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9: No bootstrap required, opened a new log
I20260808 15:56:19.816907 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:19.804304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42477 (local address 127.30.76.126:39603)
0808 15:56:19.804744 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:19.804758 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:19.804792 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:56:19.805486 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:19.805504 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:19.805826 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:19.806149 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:19.806975 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:19.808026 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:19.808912 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:19.809372 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:19.809545 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:19.812345 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:19.812350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:19.812361 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:19.812389 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:19.814107 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:19.814554 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:19.814562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:19.814571 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:19.814687 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:19.814977 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:19.814981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:19.814983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:19.815372 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:19.815608 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:19.815897 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 15:56:19.816231 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":141,"threads_started":1}
I20260808 15:56:19.819208 30122 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39603 } }
I20260808 15:56:19.819506 30122 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:19.819609 30122 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e67e3f582a46fdb3b9525c477bd9e9, State: Initialized, Role: FOLLOWER
I20260808 15:56:19.820024 30122 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39603 } }
I20260808 15:56:19.820194 30122 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:19.820230 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:19.820271 30122 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:19.820375 30122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:19.821285 30122 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39603 } }
I20260808 15:56:19.821650 30122 leader_election.cc:302] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2e67e3f582a46fdb3b9525c477bd9e9; no voters:
I20260808 15:56:19.821981 30122 leader_election.cc:288] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:19.822064 30136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:19.822335 30136 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [term 1 LEADER]: Becoming Leader. State: Replica: f2e67e3f582a46fdb3b9525c477bd9e9, State: Running, Role: LEADER
I20260808 15:56:19.822621 30136 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39603 } }
I20260808 15:56:19.823261 30122 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:19.823870 30136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39603 } } }
I20260808 15:56:19.824070 30136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:19.824339 30136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2e67e3f582a46fdb3b9525c477bd9e9. Latest consensus state: current_term: 1 leader_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e67e3f582a46fdb3b9525c477bd9e9" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39603 } } }
I20260808 15:56:19.824482 30136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:19.824795 30154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:19.828580 30154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:19.831000 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:19.822241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43487 (local address 127.30.76.126:39603)
0808 15:56:19.822362 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:19.822366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:19.822635 (+ 269us) server_negotiation.cc:407] Connection header received
0808 15:56:19.822744 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:19.822747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:19.822788 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:19.822882 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:19.823585 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:19.824386 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:19.825469 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:19.825758 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:19.825862 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:19.827439 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:19.827443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:19.827446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:19.827472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:19.829172 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:19.829656 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:19.829659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:19.829661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:19.829747 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:19.830068 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:19.830071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:19.830073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:19.830287 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:19.830425 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:19.830712 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 15:56:19.830817 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:56:19.834702 30154 catalog_manager.cc:1347] Generated new cluster ID: c4e59517599048e0bc038d0332be91ed
I20260808 15:56:19.834769 30154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:19.858064 30154 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:19.859246 30154 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:19.865732 30154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2e67e3f582a46fdb3b9525c477bd9e9: Generated new TSK 0
I20260808 15:56:19.866366 30154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:19.871129 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 30025
2026-08-08T15:56:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:19 dist-test-slave-sb2z krb5kdc[30169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:19 dist-test-slave-sb2z krb5kdc[30169](info): set up 2 sockets
Aug 08 15:56:19 dist-test-slave-sb2z krb5kdc[30169](info): commencing operation
krb5kdc: starting...
W20260808 15:56:21.099272 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:21 dist-test-slave-sb2z krb5kdc[30169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:21.222394 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:44205
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:44205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:21.335426 30188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:21.335759 30188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:21.335840 30188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:21.339960 30188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:21.340029 30188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:21.340065 30188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:21.340116 30188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:21.340153 30188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:21.345379 30188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:44205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:44205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:21.346379 30188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:21.347394 30188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:21.355552 30194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:21.355557 30193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:21.355562 30196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:21.356042 30188 server_base.cc:1034] running on GCE node
I20260808 15:56:21.356611 30188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:21.357657 30188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:21.358839 30188 hybrid_clock.cc:648] HybridClock initialized: now 1786204581358815 us; error 37 us; skew 500 ppm
Aug 08 15:56:21 dist-test-slave-sb2z krb5kdc[30169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204581, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:21.361615 30188 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:21.362524 30188 webserver.cc:460] Webserver started at http://127.30.76.126:34587/ using document root <none> and password file <none>
I20260808 15:56:21.363072 30188 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:21.363168 30188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:21.363394 30188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:21.365442 30188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "721f09b63b48472d83baa1defc34492a"
format_stamp: "Formatted at 2026-08-08 15:56:21 on dist-test-slave-sb2z"
server_key: "0da7e4c745643c1b11a6bc490f994934"
server_key_iv: "0e22242b67254a9eba06eb39cb88e412"
server_key_version: "encryptionkey@0"
I20260808 15:56:21.366020 30188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "721f09b63b48472d83baa1defc34492a"
format_stamp: "Formatted at 2026-08-08 15:56:21 on dist-test-slave-sb2z"
server_key: "0da7e4c745643c1b11a6bc490f994934"
server_key_iv: "0e22242b67254a9eba06eb39cb88e412"
server_key_version: "encryptionkey@0"
I20260808 15:56:21.369606 30188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:56:21.371903 30206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:21.373013 30188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:21.373152 30188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "721f09b63b48472d83baa1defc34492a"
format_stamp: "Formatted at 2026-08-08 15:56:21 on dist-test-slave-sb2z"
server_key: "0da7e4c745643c1b11a6bc490f994934"
server_key_iv: "0e22242b67254a9eba06eb39cb88e412"
server_key_version: "encryptionkey@0"
I20260808 15:56:21.373260 30188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:21.403066 30188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:21.405802 30188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:21.405999 30188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:21.413012 30188 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:44205
I20260808 15:56:21.413038 30279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:44205 every 8 connection(s)
I20260808 15:56:21.413954 30188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:21.416553 30280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:21.418951 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 30188
I20260808 15:56:21.419086 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:21.419377 31025 external_mini_cluster.cc:1428] Setting key 278dceed6f4e16313b8c966325b3631e
I20260808 15:56:21.422735 30280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a: Bootstrap starting.
I20260808 15:56:21.425338 30280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:21.426178 30280 log.cc:824] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:21.428054 30280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a: No bootstrap required, opened a new log
I20260808 15:56:21.429289 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:21.420772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57257 (local address 127.30.76.126:44205)
0808 15:56:21.421209 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:21.421227 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:21.421270 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:56:21.422059 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:21.422081 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:21.422507 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:21.422874 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:21.423290 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:21.424789 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:21.425816 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:21.426526 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:21.426688 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:21.427511 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:21.427517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:21.427561 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:21.427933 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:21.427984 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:21.428007 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:21.428112 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:21.428462 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 15:56:21.428628 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
I20260808 15:56:21.431488 30280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721f09b63b48472d83baa1defc34492a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44205 } }
I20260808 15:56:21.431782 30280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:21.431859 30280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721f09b63b48472d83baa1defc34492a, State: Initialized, Role: FOLLOWER
I20260808 15:56:21.431939 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:21.432302 30280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721f09b63b48472d83baa1defc34492a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44205 } }
I20260808 15:56:21.432468 30280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:21.432538 30280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:21.432642 30280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:21.433781 30280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721f09b63b48472d83baa1defc34492a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44205 } }
I20260808 15:56:21.434145 30280 leader_election.cc:302] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 721f09b63b48472d83baa1defc34492a; no voters:
I20260808 15:56:21.434533 30280 leader_election.cc:288] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:21.434623 30295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:21.434917 30295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [term 1 LEADER]: Becoming Leader. State: Replica: 721f09b63b48472d83baa1defc34492a, State: Running, Role: LEADER
I20260808 15:56:21.435257 30295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721f09b63b48472d83baa1defc34492a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44205 } }
I20260808 15:56:21.435879 30280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:21.437054 30297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "721f09b63b48472d83baa1defc34492a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721f09b63b48472d83baa1defc34492a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44205 } } }
I20260808 15:56:21.437212 30297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:21.437565 30298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 721f09b63b48472d83baa1defc34492a. Latest consensus state: current_term: 1 leader_uuid: "721f09b63b48472d83baa1defc34492a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721f09b63b48472d83baa1defc34492a" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 44205 } } }
I20260808 15:56:21.437669 30298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:21.438812 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:21.434186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39015 (local address 127.30.76.126:44205)
0808 15:56:21.434312 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:21.434316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:21.434600 (+ 284us) server_negotiation.cc:407] Connection header received
0808 15:56:21.434730 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:21.434735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:21.434785 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:21.434869 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:21.435144 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:21.435896 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:21.437012 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:21.437290 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:21.437404 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:21.438251 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:21.438254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:21.438257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:21.438269 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:21.438293 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:21.438304 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:21.438363 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:21.438635 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 15:56:21.438692 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 15:56:21.440879 30333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:21.440974 30333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:21.441078 30311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:21.441784 30311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:21.446986 30311 catalog_manager.cc:1347] Generated new cluster ID: 8822a88c09bc4fddb166e8aaa411f7b3
I20260808 15:56:21.447057 30311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:21.457028 30311 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:21.457854 30311 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:21.465813 30311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 721f09b63b48472d83baa1defc34492a: Generated new TSK 0
I20260808 15:56:21.466447 30311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:21.471014 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 30188
2026-08-08T15:56:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:21 dist-test-slave-sb2z krb5kdc[30339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:21 dist-test-slave-sb2z krb5kdc[30339](info): set up 2 sockets
Aug 08 15:56:21 dist-test-slave-sb2z krb5kdc[30339](info): commencing operation
krb5kdc: starting...
W20260808 15:56:22.686684 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:22 dist-test-slave-sb2z krb5kdc[30339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:22.811679 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45995
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:22.920651 30364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:22.920914 30364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:22.920971 30364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:22.924839 30364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:22.924890 30364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:22.924909 30364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:22.924926 30364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:22.924942 30364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:22.929915 30364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:45995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:22.930816 30364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:22.931802 30364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:22.939845 30379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:22.939833 30376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:22.939834 30374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:22.940011 30364 server_base.cc:1034] running on GCE node
I20260808 15:56:22.940660 30364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:22.941684 30364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:22.942862 30364 hybrid_clock.cc:648] HybridClock initialized: now 1786204582942841 us; error 42 us; skew 500 ppm
Aug 08 15:56:22 dist-test-slave-sb2z krb5kdc[30339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204582, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:22.945577 30364 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:22.946381 30364 webserver.cc:460] Webserver started at http://127.30.76.126:33117/ using document root <none> and password file <none>
I20260808 15:56:22.946949 30364 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:22.947036 30364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:22.947258 30364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:22.949224 30364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "7e82af19b3cc469f9ade2799b35ce670"
format_stamp: "Formatted at 2026-08-08 15:56:22 on dist-test-slave-sb2z"
server_key: "3d8c4ad7a7b8fb6994896a7d0dc37d5f"
server_key_iv: "58f61c18913877ff289c001a775256ac"
server_key_version: "encryptionkey@0"
I20260808 15:56:22.949757 30364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "7e82af19b3cc469f9ade2799b35ce670"
format_stamp: "Formatted at 2026-08-08 15:56:22 on dist-test-slave-sb2z"
server_key: "3d8c4ad7a7b8fb6994896a7d0dc37d5f"
server_key_iv: "58f61c18913877ff289c001a775256ac"
server_key_version: "encryptionkey@0"
I20260808 15:56:22.953526 30364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:56:22.956234 30391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:22.957793 30364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:56:22.957923 30364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "7e82af19b3cc469f9ade2799b35ce670"
format_stamp: "Formatted at 2026-08-08 15:56:22 on dist-test-slave-sb2z"
server_key: "3d8c4ad7a7b8fb6994896a7d0dc37d5f"
server_key_iv: "58f61c18913877ff289c001a775256ac"
server_key_version: "encryptionkey@0"
I20260808 15:56:22.958036 30364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:22.962553 30364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:22.965726 30364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:22.965914 30364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:22.973009 30364 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:45995
I20260808 15:56:22.973018 30470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:45995 every 8 connection(s)
I20260808 15:56:22.973898 30364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:22.976577 30471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:22.977111 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 30364
I20260808 15:56:22.977262 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:22.977547 31025 external_mini_cluster.cc:1428] Setting key 17a660fd8d92d143bea3405727e95775
I20260808 15:56:22.982755 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:22.978815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54355 (local address 127.30.76.126:45995)
0808 15:56:22.979226 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:22.979243 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:22.979285 (+ 42us) server_negotiation.cc:407] Connection header received
0808 15:56:22.980111 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:22.980147 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:22.980478 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:22.980818 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:22.981451 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:22.981456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:22.981470 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:22.981480 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:22.981534 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:22.981557 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:22.981699 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:22.981765 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 15:56:22.981907 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260808 15:56:22.983130 30471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670: Bootstrap starting.
I20260808 15:56:22.985514 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:22.985679 30471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:22.986536 30471 log.cc:824] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:22.988611 30471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670: No bootstrap required, opened a new log
I20260808 15:56:22.989332 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:22.987794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56027 (local address 127.30.76.126:45995)
0808 15:56:22.987903 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:22.987907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:22.988091 (+ 184us) server_negotiation.cc:407] Connection header received
0808 15:56:22.988219 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:22.988222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:22.988255 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:22.988333 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:22.988971 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:22.988975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:22.988977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:56:22.988988 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:22.989018 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:56:22.989028 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:22.989113 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:22.989189 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 15:56:22.989225 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:56:22.992338 30471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e82af19b3cc469f9ade2799b35ce670" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45995 } }
I20260808 15:56:22.992573 30471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:56:22.992657 30471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e82af19b3cc469f9ade2799b35ce670, State: Initialized, Role: FOLLOWER
I20260808 15:56:22.993085 30471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e82af19b3cc469f9ade2799b35ce670" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45995 } }
I20260808 15:56:22.993227 30471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:56:22.993295 30471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:56:22.993402 30471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:22.994387 30471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e82af19b3cc469f9ade2799b35ce670" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45995 } }
I20260808 15:56:22.994702 30471 leader_election.cc:302] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e82af19b3cc469f9ade2799b35ce670; no voters:
I20260808 15:56:22.994997 30471 leader_election.cc:288] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:56:22.995108 30486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:22.995338 30486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [term 1 LEADER]: Becoming Leader. State: Replica: 7e82af19b3cc469f9ade2799b35ce670, State: Running, Role: LEADER
I20260808 15:56:22.995667 30486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e82af19b3cc469f9ade2799b35ce670" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45995 } }
I20260808 15:56:22.996169 30471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:22.997255 30488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e82af19b3cc469f9ade2799b35ce670. Latest consensus state: current_term: 1 leader_uuid: "7e82af19b3cc469f9ade2799b35ce670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e82af19b3cc469f9ade2799b35ce670" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45995 } } }
I20260808 15:56:22.997381 30488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:22.997570 30487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e82af19b3cc469f9ade2799b35ce670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e82af19b3cc469f9ade2799b35ce670" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 45995 } } }
I20260808 15:56:22.997716 30487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670 [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:22.997838 30499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:23.001003 30499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:23.006302 30499 catalog_manager.cc:1347] Generated new cluster ID: 81da883579cc446ba186befb0354390e
I20260808 15:56:23.006366 30499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:23.023077 30499 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:23.023387 30499 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:23.029469 30499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e82af19b3cc469f9ade2799b35ce670: Generated new TSK 0
I20260808 15:56:23.030087 30499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:56:23.033701 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 30364
2026-08-08T15:56:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1561 ms)
[----------] 18 tests from AuthTokenIssuingTest (28773 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:23 dist-test-slave-sb2z krb5kdc[30517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:23 dist-test-slave-sb2z krb5kdc[30517](info): set up 2 sockets
Aug 08 15:56:23 dist-test-slave-sb2z krb5kdc[30517](info): commencing operation
krb5kdc: starting...
W20260808 15:56:24.252526 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:24.374564 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:33721
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:33721,127.30.76.125:45977,127.30.76.124:34545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:24.483098 30533 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:56:24.483418 30533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:24.483486 30533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:24.483582 30533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:24.487473 30533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:24.487572 30533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:24.487624 30533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:24.487666 30533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:24.487704 30533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:24.492743 30533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:33721,127.30.76.125:45977,127.30.76.124:34545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:33721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:24.493837 30533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:24.494846 30533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:24.503011 30539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:24.503057 30542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:24.503017 30540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:24.503307 30533 server_base.cc:1034] running on GCE node
I20260808 15:56:24.503906 30533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:24.505010 30533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:24.506202 30533 hybrid_clock.cc:648] HybridClock initialized: now 1786204584506187 us; error 37 us; skew 500 ppm
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:24.508896 30533 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:24.509786 30533 webserver.cc:460] Webserver started at http://127.30.76.126:38641/ using document root <none> and password file <none>
I20260808 15:56:24.510324 30533 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:24.510407 30533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:24.510635 30533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:24.512693 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "ba91ba35c2314ca18084aee8c1bda70d"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "48641140f74b35ae109d35d953bf2141"
server_key_iv: "be4ac12fb1c8c7ff2210160ba2f0162b"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.513249 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "ba91ba35c2314ca18084aee8c1bda70d"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "48641140f74b35ae109d35d953bf2141"
server_key_iv: "be4ac12fb1c8c7ff2210160ba2f0162b"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.516849 30533 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:56:24.519079 30552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:24.520294 30533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:24.520429 30533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "ba91ba35c2314ca18084aee8c1bda70d"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "48641140f74b35ae109d35d953bf2141"
server_key_iv: "be4ac12fb1c8c7ff2210160ba2f0162b"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.520536 30533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:24.538880 30533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:24.541785 30533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:24.541975 30533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:24.548772 30533 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:33721
I20260808 15:56:24.549059 30634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:33721 every 8 connection(s)
I20260808 15:56:24.549695 30533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:24.550305 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 30533
I20260808 15:56:24.550453 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:24.550741 31025 external_mini_cluster.cc:1428] Setting key 624e3b6add611f843ab71ff379950b6b
I20260808 15:56:24.552603 30635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:24.556648 30635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.76.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.125@KRBTEST.COM" created.
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:56:24.572762 30641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.558809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.30.76.126:33721)
0808 15:56:24.559375 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:24.559386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:24.559429 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:56:24.560477 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:24.560481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:24.560729 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:24.560885 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:24.561666 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.562826 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.564379 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.565024 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.565281 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.567622 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:24.567641 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:24.567657 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:24.567692 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:24.569779 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.570259 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.570265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.570267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.570405 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.570707 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.570711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.570713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.571248 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:24.571384 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:24.571798 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 15:56:24.571999 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":441,"spinlock_wait_cycles":4864,"thread_start_us":165,"threads_started":1}
I20260808 15:56:24.572767 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.558809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:33721 (local address 127.0.0.1:34124)
0808 15:56:24.559262 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.559315 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:24.560195 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:24.560917 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:24.560929 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:24.561240 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:24.561481 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.561494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.562954 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.562961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.564245 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.564261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.565176 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.565178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.565305 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.565977 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:24.565993 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:24.567438 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:24.569887 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.569895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.569903 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.570122 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.570503 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.570505 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.570506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.570610 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.571371 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:24.571376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:24.571496 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 15:56:24.571691 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 15:56:24.571943 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":8,"spinlock_wait_cycles":6272,"thread_start_us":127,"threads_started":1}
I20260808 15:56:24.575071 30635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:24.575794 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.575570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46434)
0808 15:56:24.575675 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.575742 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 15:56:24.576011 30558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
I20260808 15:56:24.576891 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.576709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46450)
0808 15:56:24.576799 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.576843 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 15:56:24.577021 30635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:45977: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab.
Entry for principal kudu/127.30.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab.
I20260808 15:56:24.578799 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:45977
--webserver_interface=127.30.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab
--principal=kudu/127.30.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:33721,127.30.76.125:45977,127.30.76.124:34545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:56:24.626223 30635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } attempt: 1
I20260808 15:56:24.627092 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.626787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46456)
0808 15:56:24.626945 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.627013 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 15:56:24.627913 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.627688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46470)
0808 15:56:24.627799 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.627860 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:56:24.628067 30635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:45977: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
W20260808 15:56:24.693125 30644 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:56:24.693403 30644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:24.693426 30644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:24.693529 30644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 15:56:24.696215 30635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } attempt: 2
I20260808 15:56:24.697026 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.696729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46474)
0808 15:56:24.696882 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.696951 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 15:56:24.697814 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.697603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46478)
0808 15:56:24.697706 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.697753 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:56:24.697923 30644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:24.698001 30644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:24.698037 30644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:24.698017 30635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:45977: Network error: Client connection negotiation failed: client connection to 127.30.76.125:45977: connect: Connection refused (error 111)
W20260808 15:56:24.698073 30644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:24.698117 30644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:24.703570 30644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:33721,127.30.76.125:45977,127.30.76.124:34545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab
--principal=kudu/127.30.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:45977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:24.704630 30644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:24.705647 30644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:24.714174 30654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:24.714180 30656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:24.714180 30653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:24.714362 30644 server_base.cc:1034] running on GCE node
I20260808 15:56:24.715059 30644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:24.716204 30644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:24.717391 30644 hybrid_clock.cc:648] HybridClock initialized: now 1786204584717376 us; error 27 us; skew 500 ppm
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:24.720239 30644 init.cc:377] Logged in from keytab as kudu/127.30.76.125@KRBTEST.COM (short username kudu)
I20260808 15:56:24.721089 30644 webserver.cc:460] Webserver started at http://127.30.76.125:36971/ using document root <none> and password file <none>
I20260808 15:56:24.721617 30644 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:24.721686 30644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:24.721860 30644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:24.724386 30644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data/instance:
uuid: "4d6c8d478a32471abc2163a89a87d723"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "0bdf79ec13586700fb3d09623227833a"
server_key_iv: "6987a88e1d4f203e7aa73529adb591bd"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.724979 30644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal/instance:
uuid: "4d6c8d478a32471abc2163a89a87d723"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "0bdf79ec13586700fb3d09623227833a"
server_key_iv: "6987a88e1d4f203e7aa73529adb591bd"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.728696 30644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:56:24.731074 30665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:24.732360 30644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:24.732475 30644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal
uuid: "4d6c8d478a32471abc2163a89a87d723"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "0bdf79ec13586700fb3d09623227833a"
server_key_iv: "6987a88e1d4f203e7aa73529adb591bd"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.732563 30644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:24.754729 30644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:24.758095 30644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:24.758261 30644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:24.765089 30644 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.125:45977
I20260808 15:56:24.765393 30741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.125:45977 every 8 connection(s)
I20260808 15:56:24.766173 30644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
I20260808 15:56:24.768994 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:24.772830 30742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:24.775697 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 30644
I20260808 15:56:24.775833 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-1/wal/instance
I20260808 15:56:24.776118 31025 external_mini_cluster.cc:1428] Setting key 21f553c639724d2ad1172348180da910
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.124@KRBTEST.COM; defaulting to no policy
I20260808 15:56:24.787042 30641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.774686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34136 (local address 127.30.76.126:33721)
0808 15:56:24.774826 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:24.774830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:24.775715 (+ 885us) server_negotiation.cc:407] Connection header received
0808 15:56:24.776732 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:24.776735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:24.776794 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:24.776885 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:24.778086 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.778610 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.780079 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.780517 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.780651 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.783495 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:24.783498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:24.783500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:24.783556 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:24.784992 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.785635 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.785638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.785639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.785713 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.786017 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.786019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.786020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.786216 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:24.786291 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:24.786822 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 15:56:24.786919 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.124@KRBTEST.COM" created.
I20260808 15:56:24.788008 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.775075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:33721 (local address 127.0.0.1:34136)
0808 15:56:24.775572 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.775620 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:24.776521 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:24.777052 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:24.777059 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:24.777402 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:24.777892 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.777912 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.778748 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.778754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.779947 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.779957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.780669 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.780672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.780806 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.781447 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:24.781465 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:24.783318 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:24.785118 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.785127 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.785142 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.785524 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.785800 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.785803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.785804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.785939 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.786317 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:24.786326 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:24.786602 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 15:56:24.786766 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 15:56:24.787004 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":122,"threads_started":1}
I20260808 15:56:24.789170 30635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } attempt: 3
I20260808 15:56:24.789460 30742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
I20260808 15:56:24.801517 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.789587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46484)
0808 15:56:24.789694 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.789711 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:24.789809 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:24.790934 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:24.790939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:24.791127 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:24.791262 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.791270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.792187 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.792192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.793098 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.793106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.793713 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.793717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.793973 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.794472 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:24.794483 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:24.796821 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:24.799774 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.799777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.799779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.800050 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.800344 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.800348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.800349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.800441 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.800958 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:24.800962 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:24.801095 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 15:56:24.801207 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 15:56:24.801349 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 15:56:24.801549 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.789971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46500)
0808 15:56:24.790086 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.790100 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:24.790172 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:24.790856 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:24.790859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:24.791031 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:24.791124 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.791130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.792166 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.792172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.793134 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:24.793149 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.793941 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.793944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:24.794061 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.794656 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:24.794673 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:24.796751 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:24.799637 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.799639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.799641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.799903 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.800234 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:24.800237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:24.800238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:24.800342 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:24.801042 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:24.801045 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:24.801122 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 15:56:24.801231 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 15:56:24.801354 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 15:56:24.801856 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.790117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.30.76.125:45977)
0808 15:56:24.790512 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:24.790519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:24.790556 (+ 37us) server_negotiation.cc:407] Connection header received
0808 15:56:24.790587 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:24.790589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:24.790740 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:24.790877 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:24.791231 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.792051 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.793269 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.793729 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.793876 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.796915 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:24.796926 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:24.796942 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:24.796968 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:24.799446 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.800028 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.800033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.800035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.800137 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.800435 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.800437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.800438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.800845 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:24.800999 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:24.801312 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 15:56:24.801508 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":340,"spinlock_wait_cycles":9344,"thread_start_us":151,"threads_started":1}
I20260808 15:56:24.801919 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.789710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.30.76.125:45977)
0808 15:56:24.790490 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:24.790512 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:24.790537 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:56:24.790583 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:24.790589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:24.790772 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:24.791386 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:24.791503 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.792074 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.793260 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:24.793591 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:24.793699 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:24.796950 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:24.796953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:24.796955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:24.796978 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:24.799682 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.800153 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.800159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.800163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.800246 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:24.800590 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:24.800593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:24.800595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:24.800843 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:24.800996 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:24.801310 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 15:56:24.801761 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":669,"spinlock_wait_cycles":40192,"thread_start_us":111,"threads_started":1}
I20260808 15:56:24.803575 30635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:24.803643 30742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:24.804301 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.804064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48828)
0808 15:56:24.804174 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.804231 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 15:56:24.804373 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.804106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48832)
0808 15:56:24.804227 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.804298 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:56:24.804555 30670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
I20260808 15:56:24.805121 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.804843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48834)
0808 15:56:24.805007 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.805059 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 15:56:24.805327 30635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:34545: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
I20260808 15:56:24.805446 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.805241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48850)
0808 15:56:24.805333 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.805386 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 15:56:24.805569 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:34545: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab.
Entry for principal kudu/127.30.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab.
I20260808 15:56:24.807147 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:34545
--webserver_interface=127.30.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab
--principal=kudu/127.30.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:34049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:33721,127.30.76.125:45977,127.30.76.124:34545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:56:24.836490 30635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } attempt: 1
I20260808 15:56:24.837319 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.837012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48866)
0808 15:56:24.837184 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.837242 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 15:56:24.838002 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.837794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48880)
0808 15:56:24.837897 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.837946 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 15:56:24.838145 30635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:34545: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
I20260808 15:56:24.854820 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } attempt: 1
I20260808 15:56:24.855674 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.855364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48896)
0808 15:56:24.855504 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.855599 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 15:56:24.856338 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.856139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48908)
0808 15:56:24.856243 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.856292 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:56:24.856459 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:34545: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
I20260808 15:56:24.913326 30635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } attempt: 2
I20260808 15:56:24.914199 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.913857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48918)
0808 15:56:24.914056 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.914116 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 15:56:24.914932 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.914695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48928)
0808 15:56:24.914809 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.914864 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:56:24.915092 30635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:34545: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
W20260808 15:56:24.920204 30755 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:56:24.920519 30755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:24.920586 30755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:24.920670 30755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 15:56:24.924597 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } attempt: 2
W20260808 15:56:24.925240 30755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:24.925313 30755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260808 15:56:24.925300 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.925052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48938)
0808 15:56:24.925173 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.925232 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:56:24.925379 30755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:24.925431 30755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:24.925467 30755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:24.925999 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.925791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48952)
0808 15:56:24.925893 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:24.925941 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:56:24.926229 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:34545: Network error: Client connection negotiation failed: client connection to 127.30.76.124:34545: connect: Connection refused (error 111)
I20260808 15:56:24.931836 30755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:34049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:33721,127.30.76.125:45977,127.30.76.124:34545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab
--principal=kudu/127.30.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:34545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:24.932889 30755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:24.933895 30755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:24.942354 30762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:24.942397 30764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:24.942373 30761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:24.942626 30755 server_base.cc:1034] running on GCE node
I20260808 15:56:24.943382 30755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:24.944511 30755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:24.945688 30755 hybrid_clock.cc:648] HybridClock initialized: now 1786204584945671 us; error 25 us; skew 500 ppm
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:24.948696 30755 init.cc:377] Logged in from keytab as kudu/127.30.76.124@KRBTEST.COM (short username kudu)
I20260808 15:56:24.949551 30755 webserver.cc:460] Webserver started at http://127.30.76.124:40051/ using document root <none> and password file <none>
I20260808 15:56:24.950116 30755 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:24.950207 30755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:24.950431 30755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:24.952498 30755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data/instance:
uuid: "7146e496acc34742a13c3835213ddd1b"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "ae1ef52074e974e58f1f3032da7d5031"
server_key_iv: "39b498af0d4724f7528438f5b944f342"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.953115 30755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal/instance:
uuid: "7146e496acc34742a13c3835213ddd1b"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "ae1ef52074e974e58f1f3032da7d5031"
server_key_iv: "39b498af0d4724f7528438f5b944f342"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.956815 30755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:56:24.959178 30773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:24.960425 30755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:56:24.960579 30755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal
uuid: "7146e496acc34742a13c3835213ddd1b"
format_stamp: "Formatted at 2026-08-08 15:56:24 on dist-test-slave-sb2z"
server_key: "ae1ef52074e974e58f1f3032da7d5031"
server_key_iv: "39b498af0d4724f7528438f5b944f342"
server_key_version: "encryptionkey@0"
I20260808 15:56:24.960686 30755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:24.979471 30755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:24.983100 30755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:24.983318 30755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:24.990299 30755 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.124:34545
I20260808 15:56:24.990594 30851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.124:34545 every 8 connection(s)
I20260808 15:56:24.991267 30755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
I20260808 15:56:24.993600 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 30755
I20260808 15:56:24.993747 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786204294980517-31025-0/minicluster-data/master-2/wal/instance
I20260808 15:56:24.994071 31025 external_mini_cluster.cc:1428] Setting key 8434df0a5ec35ecfa5351a18f0577a1b
I20260808 15:56:24.994160 30852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:24.999455 30852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
Aug 08 15:56:24 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:56:25.010035 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:24.995506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.30.76.124:34545)
0808 15:56:24.995963 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:24.995982 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:24.996022 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:56:24.997003 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:24.997026 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:24.997428 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:24.997800 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:24.998519 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.000061 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.000957 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.001524 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.001748 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.004708 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.004719 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.004729 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.004756 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.006776 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.007270 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.007276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.007282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.007393 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.007838 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.007843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.007845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.008384 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.008605 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.008872 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.009178 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":104,"threads_started":1}
I20260808 15:56:25.011785 30641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.001435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.30.76.126:33721)
0808 15:56:25.001597 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.001602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.001828 (+ 226us) server_negotiation.cc:407] Connection header received
0808 15:56:25.002032 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.002036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.002099 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.002174 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.002897 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.003700 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.004732 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.004944 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.005125 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.007511 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.007516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.007519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.007587 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.009199 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.009660 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.009669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.009670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.009747 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.010059 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.010064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.010066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.011037 (+ 971us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.011154 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.011480 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.011624 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:56:25.011809 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.001333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:33721 (local address 127.0.0.1:34140)
0808 15:56:25.001710 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:25.001758 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:25.001914 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:25.002250 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:25.002258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:25.002640 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:25.002767 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.002773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.003826 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.003829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.004601 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.004611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.005113 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.005116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.005270 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.005787 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:25.005807 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:25.007363 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:25.009302 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.009310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.009314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.009546 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.009840 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.009844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.009846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.009928 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.011153 (+ 1225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:25.011159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:25.011268 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 15:56:25.011414 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 15:56:25.011625 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":156,"threads_started":1}
I20260808 15:56:25.012908 30852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:25.014248 30635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } attempt: 3
I20260808 15:56:25.017396 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } attempt: 3
Aug 08 15:56:25 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
Aug 08 15:56:25 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
Aug 08 15:56:25 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:56:25.023366 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.013538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.30.76.125:45977)
0808 15:56:25.013672 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.013677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.013691 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:56:25.013732 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.013735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.013779 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.013860 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.014508 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.015242 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.016229 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.016465 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.016570 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.018558 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.018561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.018564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.018588 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.020344 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.020884 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.020890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.020895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.021007 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.021613 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.021617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.021619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.021841 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.021912 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.023063 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.023205 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:56:25.023928 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.013411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:45977 (local address 127.0.0.1:46514)
0808 15:56:25.013553 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:25.013574 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:25.013651 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:25.013886 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:25.013888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:25.014064 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:25.014151 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.014156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.015347 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.015351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.016130 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.016140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.016574 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.016578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.016669 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.017117 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:25.017130 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:25.018426 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:25.020460 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.020466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.020470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.020752 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.021145 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.021151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.021154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.021470 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.022748 (+ 1278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:25.022751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:25.022857 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 15:56:25.023636 (+ 779us) client_negotiation.cc:240] Negotiation successful
0808 15:56:25.023780 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 15:56:25.024673 30852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:25.024935 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.014635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48978)
0808 15:56:25.014788 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:25.014800 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:25.014877 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:25.015082 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:25.015085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:25.015255 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:25.015367 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.015373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.016260 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.016263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.017471 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.017484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.018069 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.018074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.018180 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.018911 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:25.018922 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:25.020497 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:25.023187 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.023193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.023196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.023447 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.023825 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.023833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.023837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.023935 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.024413 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:25.024419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:25.024524 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 15:56:25.024638 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 15:56:25.024768 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
Aug 08 15:56:25 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:56:25.025512 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.014691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.30.76.124:34545)
0808 15:56:25.014785 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.014788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.014844 (+ 56us) server_negotiation.cc:407] Connection header received
0808 15:56:25.014939 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.014941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.014977 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.015053 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.015484 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.016129 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.017617 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.017937 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.018044 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.020642 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.020646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.020648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.020673 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.023056 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.023612 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.023616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.023619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.023710 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.024071 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.024074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.024076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.024262 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.024358 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.025233 (+ 875us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.025351 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":38}
I20260808 15:56:25.025898 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.012230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.30.76.126:33721)
0808 15:56:25.012615 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.012622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.012632 (+ 10us) server_negotiation.cc:407] Connection header received
0808 15:56:25.012660 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.012662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.012694 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.012784 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.013546 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.014222 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.015630 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.015989 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.016103 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.020053 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.020056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.020059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.020089 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.022176 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.022667 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.022672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.022677 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.022770 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.023137 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.023142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.023146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.023343 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.025288 (+ 1945us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.025373 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.025506 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":125,"threads_started":1}
I20260808 15:56:25.029136 30635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: Bootstrap starting.
I20260808 15:56:25.029913 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.017776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48990)
0808 15:56:25.018036 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:25.018049 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:25.018116 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:25.019293 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:25.019296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:25.020542 (+ 1246us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:25.020647 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.020653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.021761 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.021764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.022607 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.022621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.023273 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.023278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.023378 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.023889 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:25.023903 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:25.025421 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:25.027989 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.027992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.027997 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.028259 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.028618 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.028621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.028623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.028721 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.029431 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:25.029434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:25.029550 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 15:56:25.029637 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 15:56:25.029760 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
Aug 08 15:56:25 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:56:25.030591 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.018294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.30.76.124:34545)
0808 15:56:25.018753 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.018762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.018776 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:56:25.018814 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.018819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.019137 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.019720 (+ 583us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.020771 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.021641 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.022730 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.023144 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.025563 (+ 2419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.025841 (+ 278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.025844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.025846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.025861 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.027880 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.028396 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.028401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.028404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.028495 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.028841 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.028844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.028845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.029318 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.030166 (+ 848us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.030252 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.030373 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"spinlock_wait_cycles":18304,"thread_start_us":105,"threads_started":1}
I20260808 15:56:25.032968 30635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:25.033697 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: Bootstrap starting.
I20260808 15:56:25.034011 30635 log.cc:824] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:25.034096 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.025315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.30.76.124:34545)
0808 15:56:25.025666 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.025670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.025682 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:56:25.025710 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.025712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.025752 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.025828 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.026613 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.027239 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.028233 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.028524 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.028596 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.030665 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.030668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.030671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.030696 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.032409 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.032849 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.032855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.032858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.032941 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.033266 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.033269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.033271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.033486 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.033555 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.033845 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.033948 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":113,"threads_started":1}
I20260808 15:56:25.034528 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.025146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:34545 (local address 127.0.0.1:48998)
0808 15:56:25.025534 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:25.025547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:25.025629 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:25.026237 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:25.026240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:25.026403 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:25.026495 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.026502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.027358 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.027361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.028110 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:25.028121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.028713 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.028716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:25.028780 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.029193 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:25.029210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:25.030530 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:25.032528 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.032531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.032533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.032747 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.033049 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:25.033052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:25.033053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:25.033139 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:25.033612 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:25.033614 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:25.033686 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 15:56:25.034244 (+ 558us) client_negotiation.cc:240] Negotiation successful
0808 15:56:25.034368 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333}
I20260808 15:56:25.036125 30742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:25.036881 30742 log.cc:824] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:25.037089 30635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: No bootstrap required, opened a new log
I20260808 15:56:25.037171 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: Bootstrap starting.
Aug 08 15:56:25 dist-test-slave-sb2z krb5kdc[30517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
I20260808 15:56:25.039546 30852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:25.039755 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: No bootstrap required, opened a new log
I20260808 15:56:25.040313 30852 log.cc:824] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:25.041266 30635 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
I20260808 15:56:25.041517 30635 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba91ba35c2314ca18084aee8c1bda70d, State: Initialized, Role: FOLLOWER
I20260808 15:56:25.041869 30635 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
I20260808 15:56:25.042387 30868 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.042603 30868 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.043325 30635 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:25.043380 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.027352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46528 (local address 127.30.76.125:45977)
0808 15:56:25.027460 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.027463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.027775 (+ 312us) server_negotiation.cc:407] Connection header received
0808 15:56:25.027816 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.027819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.027861 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.027930 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.028787 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.029512 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.030523 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.030821 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.030953 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.039226 (+ 8273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.039230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.039232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.039257 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.041137 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.041688 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.041692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.041694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.041781 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.042104 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.042107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.042110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.042331 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.042510 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.043128 (+ 618us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.043240 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 15:56:25.043416 30560 tablet.cc:2369] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:56:25.043633 30742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
I20260808 15:56:25.043833 30742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6c8d478a32471abc2163a89a87d723, State: Initialized, Role: FOLLOWER
I20260808 15:56:25.044144 30742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
I20260808 15:56:25.045286 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: No bootstrap required, opened a new log
I20260808 15:56:25.045328 30742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:25.048167 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:25.048454 30876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.048561 30876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.049044 30852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
I20260808 15:56:25.049257 30852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7146e496acc34742a13c3835213ddd1b, State: Initialized, Role: FOLLOWER
I20260808 15:56:25.049580 30852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
I20260808 15:56:25.049713 30591 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ba91ba35c2314ca18084aee8c1bda70d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34148
I20260808 15:56:25.049999 30591 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 15:56:25.050117 30591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:25.050797 30852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:25.051360 30591 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
W20260808 15:56:25.051690 30901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:25.051764 30901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:25.052309 30591 leader_election.cc:288] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [CANDIDATE]: Term 1 election: Requested vote from peers 4d6c8d478a32471abc2163a89a87d723 (127.30.76.125:45977), 7146e496acc34742a13c3835213ddd1b (127.30.76.124:34545)
I20260808 15:56:25.053328 30812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba91ba35c2314ca18084aee8c1bda70d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7146e496acc34742a13c3835213ddd1b"
I20260808 15:56:25.053576 30812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:25.054841 30812 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba91ba35c2314ca18084aee8c1bda70d in term 1.
I20260808 15:56:25.052449 30533 master_runner.cc:186] Error getting master registration for 127.30.76.124:34545: OK, instance_id { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" instance_seqno: 1786204584988320 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:56:25.055044 30533 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:56:25.055233 30556 leader_election.cc:302] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7146e496acc34742a13c3835213ddd1b, ba91ba35c2314ca18084aee8c1bda70d; no voters:
I20260808 15:56:25.055642 30702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba91ba35c2314ca18084aee8c1bda70d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4d6c8d478a32471abc2163a89a87d723"
I20260808 15:56:25.055872 30702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:25.057133 30702 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba91ba35c2314ca18084aee8c1bda70d in term 1.
W20260808 15:56:25.058790 30886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:25.058868 30886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:25.059105 30902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.059229 30902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.060741 30868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:25.061470 30755 master_runner.cc:263] No leader master found from master 7146e496acc34742a13c3835213ddd1b
I20260808 15:56:25.061542 30755 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:56:25.063696 30644 master_runner.cc:263] No leader master found from master 4d6c8d478a32471abc2163a89a87d723
I20260808 15:56:25.063771 30644 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:56:25.064322 30868 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [term 1 LEADER]: Becoming Leader. State: Replica: ba91ba35c2314ca18084aee8c1bda70d, State: Running, Role: LEADER
I20260808 15:56:25.064801 30868 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } }
W20260808 15:56:25.066936 30920 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 15:56:25.066906 30868 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba91ba35c2314ca18084aee8c1bda70d. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
W20260808 15:56:25.067022 30920 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:25.067051 30868 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:25.071877 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.060120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.30.76.126:33721)
0808 15:56:25.060227 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.060231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.060857 (+ 626us) server_negotiation.cc:407] Connection header received
0808 15:56:25.061106 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.061109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.061159 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.061216 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.062373 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.063175 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.064712 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.065138 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.065221 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.067385 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.067389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.067393 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.067422 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.069903 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.070422 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.070426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.070428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.070515 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.070842 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.070845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.070847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.071082 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.071199 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.071590 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.071715 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 15:56:25.076567 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.061140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.30.76.124:34545)
0808 15:56:25.061326 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.061331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.067494 (+ 6163us) server_negotiation.cc:407] Connection header received
0808 15:56:25.067703 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.067706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.067756 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.067809 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.068676 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.069442 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.070659 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.071022 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.071100 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.072855 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.072859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.072865 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.072896 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.074642 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.075142 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.075146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.075148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.075241 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.075568 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.075574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.075579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.075812 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.075938 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.076293 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.076413 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260808 15:56:25.083369 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:25.061201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.30.76.125:45977)
0808 15:56:25.061411 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.061415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.071800 (+ 10385us) server_negotiation.cc:407] Connection header received
0808 15:56:25.072001 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.072005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.072061 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.072116 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.075623 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.076395 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.077750 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:25.078105 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:25.078174 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:25.079908 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:25.079912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:25.079915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:25.079942 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:25.081582 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.082067 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.082071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.082072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.082159 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:25.082440 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:25.082443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:25.082445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:25.082656 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:25.082760 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:25.083102 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 15:56:25.083215 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260808 15:56:25.083806 30941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:25.084908 30941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:25.092197 30812 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [term 1 FOLLOWER]: Refusing update from remote peer ba91ba35c2314ca18084aee8c1bda70d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:56:25.093592 30937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:56:25.098798 30902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba91ba35c2314ca18084aee8c1bda70d. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.098917 30902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.100931 30702 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [term 1 FOLLOWER]: Refusing update from remote peer ba91ba35c2314ca18084aee8c1bda70d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:56:25.101608 30937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:56:25.105533 30876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba91ba35c2314ca18084aee8c1bda70d. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.105660 30876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.111685 30937 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.111815 30937 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:25.113070 30902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.113189 30902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.114910 30943 mvcc.cc:205] Tried to move back new op lower bound from 7316293980524613632 to 7316293980430110720. Current Snapshot: MvccSnapshot[applied={T|T < 7316293980524613632}]
I20260808 15:56:25.115623 30942 mvcc.cc:205] Tried to move back new op lower bound from 7316293980524613632 to 7316293980430110720. Current Snapshot: MvccSnapshot[applied={T|T < 7316293980524613632}]
I20260808 15:56:25.119078 30868 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.119204 30868 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:25.119150 30941 catalog_manager.cc:1347] Generated new cluster ID: 999e3b6f843e497fb56712f765720c2a
I20260808 15:56:25.119315 30941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:25.123118 30876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba91ba35c2314ca18084aee8c1bda70d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba91ba35c2314ca18084aee8c1bda70d" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 33721 } } peers { permanent_uuid: "4d6c8d478a32471abc2163a89a87d723" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 45977 } } peers { permanent_uuid: "7146e496acc34742a13c3835213ddd1b" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 34545 } } }
I20260808 15:56:25.123239 30876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:25.123591 30946 mvcc.cc:205] Tried to move back new op lower bound from 7316293980524613632 to 7316293980430110720. Current Snapshot: MvccSnapshot[applied={T|T < 7316293980524613632}]
I20260808 15:56:25.155642 30941 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:25.157069 30941 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:25.177294 30941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba91ba35c2314ca18084aee8c1bda70d: Generated new TSK 0
I20260808 15:56:25.180485 30941 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:56:25.222450 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.219722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.30.76.125:45977)
0808 15:56:25.219860 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.219865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.221243 (+ 1378us) server_negotiation.cc:407] Connection header received
0808 15:56:25.221491 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.221495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.221550 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.221611 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.222304 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46552: BlockingRecv error: recv got EOF from 127.0.0.1:46552 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 15:56:25.224264 30855 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.220586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.30.76.124:34545)
0808 15:56:25.220704 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.220711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.222859 (+ 2148us) server_negotiation.cc:407] Connection header received
0808 15:56:25.223352 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.223356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.223411 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.223482 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.224145 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49014: BlockingRecv error: recv got EOF from 127.0.0.1:49014 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 15:56:25.225898 30858 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.224854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34164 (local address 127.30.76.126:33721)
0808 15:56:25.224983 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.224988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.225005 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:56:25.225137 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.225140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.225192 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.225247 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.225782 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34164: BlockingRecv error: recv got EOF from 127.0.0.1:34164 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 15:56:25.230708 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.229079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.30.76.125:45977)
0808 15:56:25.229185 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.229190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.229649 (+ 459us) server_negotiation.cc:407] Connection header received
0808 15:56:25.229885 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.229889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.229970 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.230038 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.230593 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46562: BlockingRecv error: recv got EOF from 127.0.0.1:46562 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 15:56:25.236058 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.234244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.30.76.125:45977)
0808 15:56:25.234357 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.234363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.234907 (+ 544us) server_negotiation.cc:407] Connection header received
0808 15:56:25.235092 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.235095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.235142 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.235193 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.235952 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46564: BlockingRecv error: recv got EOF from 127.0.0.1:46564 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 15:56:25.241699 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.240050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.30.76.125:45977)
0808 15:56:25.240147 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.240151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.240749 (+ 598us) server_negotiation.cc:407] Connection header received
0808 15:56:25.240960 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.240964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.241016 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.241071 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.241604 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46574: BlockingRecv error: recv got EOF from 127.0.0.1:46574 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 15:56:25.248977 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.247475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.30.76.125:45977)
0808 15:56:25.247618 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.247622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.248136 (+ 514us) server_negotiation.cc:407] Connection header received
0808 15:56:25.248310 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.248313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.248356 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.248404 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.248882 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46588: BlockingRecv error: recv got EOF from 127.0.0.1:46588 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 15:56:25.260483 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.258896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.30.76.125:45977)
0808 15:56:25.258994 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.258998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.259508 (+ 510us) server_negotiation.cc:407] Connection header received
0808 15:56:25.259733 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.259736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.259789 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.259841 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.260375 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46598: BlockingRecv error: recv got EOF from 127.0.0.1:46598 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 15:56:25.280138 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.278596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.30.76.125:45977)
0808 15:56:25.278703 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.278707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.279327 (+ 620us) server_negotiation.cc:407] Connection header received
0808 15:56:25.279447 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.279449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.279495 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.279589 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.280029 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46602: BlockingRecv error: recv got EOF from 127.0.0.1:46602 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 15:56:25.315474 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.314292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.30.76.125:45977)
0808 15:56:25.314448 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.314453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.314608 (+ 155us) server_negotiation.cc:407] Connection header received
0808 15:56:25.314777 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.314785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.314832 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.314918 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.315370 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46610: BlockingRecv error: recv got EOF from 127.0.0.1:46610 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 15:56:25.382887 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.381712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46618 (local address 127.30.76.125:45977)
0808 15:56:25.381872 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.381878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.382020 (+ 142us) server_negotiation.cc:407] Connection header received
0808 15:56:25.382151 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.382154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.382204 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.382280 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.382730 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46618: BlockingRecv error: recv got EOF from 127.0.0.1:46618 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 15:56:25.514588 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.513302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.30.76.125:45977)
0808 15:56:25.513527 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.513533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.513645 (+ 112us) server_negotiation.cc:407] Connection header received
0808 15:56:25.513825 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.513828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.513888 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.513979 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.514451 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46630: BlockingRecv error: recv got EOF from 127.0.0.1:46630 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 15:56:25.774415 30753 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:25.773162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.30.76.125:45977)
0808 15:56:25.773287 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:25.773293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:25.773405 (+ 112us) server_negotiation.cc:407] Connection header received
0808 15:56:25.773571 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:25.773573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:25.773627 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:25.773697 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:25.774296 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46634: BlockingRecv error: recv got EOF from 127.0.0.1:46634 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 15:56:26.053176 30901 catalog_manager.cc:1259] Loaded cluster ID: 999e3b6f843e497fb56712f765720c2a
I20260808 15:56:26.053261 30901 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: loading cluster ID for follower catalog manager: success
I20260808 15:56:26.054922 30901 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: acquiring CA information for follower catalog manager: success
I20260808 15:56:26.056115 30901 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d6c8d478a32471abc2163a89a87d723: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:56:26.068104 30920 catalog_manager.cc:1259] Loaded cluster ID: 999e3b6f843e497fb56712f765720c2a
I20260808 15:56:26.068176 30920 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: loading cluster ID for follower catalog manager: success
I20260808 15:56:26.069751 30920 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: acquiring CA information for follower catalog manager: success
I20260808 15:56:26.070835 30920 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7146e496acc34742a13c3835213ddd1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:56:26.292363 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:26.288764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.30.76.125:45977)
0808 15:56:26.289005 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:26.289009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:26.289026 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:56:26.289181 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:26.289183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:26.289313 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:26.289423 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:26.289646 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.290160 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.290960 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.291184 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.291260 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:26.291570 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:26.292013 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:26.292107 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:26.292161 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 15:56:26.292208 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260808 15:56:26.298904 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:26.295353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49028 (local address 127.30.76.124:34545)
0808 15:56:26.295616 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:26.295620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:26.295637 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:56:26.295802 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:26.295805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:26.295912 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:26.296002 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:26.296182 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.296671 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.297518 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.297749 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.297860 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:26.298146 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:26.298503 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:26.298634 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:26.298702 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 15:56:26.298755 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":64,"threads_started":1}
I20260808 15:56:26.306268 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:26.301824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.30.76.126:33721)
0808 15:56:26.302232 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:26.302236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:26.302251 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:56:26.302337 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:26.302340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:26.302443 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:26.302586 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:26.302828 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.303764 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.304764 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.305002 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.305106 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:26.305445 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:26.305770 (+ 325us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:26.305859 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:26.306023 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 15:56:26.306097 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":79,"threads_started":1}
I20260808 15:56:26.306421 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:26.302130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.30.76.124:34545)
0808 15:56:26.302250 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:26.302254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:26.302702 (+ 448us) server_negotiation.cc:407] Connection header received
0808 15:56:26.302852 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:26.302855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:26.302908 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:26.302960 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:26.303286 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.304074 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.305243 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.305517 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.305625 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:26.305901 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:26.305988 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:26.306056 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:26.306204 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 15:56:26.306269 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:56:26.306577 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:26.301696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.30.76.125:45977)
0808 15:56:26.302058 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:26.302061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:26.302363 (+ 302us) server_negotiation.cc:407] Connection header received
0808 15:56:26.302479 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:26.302482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:26.302544 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:26.302613 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:26.303023 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.303763 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.305135 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:26.305464 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:26.305537 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:26.305996 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:26.306070 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:26.306119 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:26.306396 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 15:56:26.306452 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260808 15:56:26.314214 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 30533
I20260808 15:56:26.326237 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 30644
I20260808 15:56:26.335906 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 30755
2026-08-08T15:56:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3303 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:56:26 dist-test-slave-sb2z krb5kdc[31083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:56:26 dist-test-slave-sb2z krb5kdc[31083](info): set up 2 sockets
Aug 08 15:56:26 dist-test-slave-sb2z krb5kdc[31083](info): commencing operation
krb5kdc: starting...
W20260808 15:56:27.566015 31025 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/test-user.keytab.
Aug 08 15:56:27 dist-test-slave-sb2z krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:56:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
Entry for principal kudu/127.30.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab.
I20260808 15:56:27.686475 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39573
--webserver_interface=127.30.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39573,127.30.76.125:40409,127.30.76.124:44999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:56:27.796640 31102 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:56:27.796937 31102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:27.797005 31102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:27.797083 31102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:27.801110 31102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:27.801185 31102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:27.801222 31102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:27.801272 31102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:27.801313 31102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:27.806301 31102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39573,127.30.76.125:40409,127.30.76.124:44999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.126.keytab
--principal=kudu/127.30.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.126:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:27.807277 31102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:27.808300 31102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:27.816429 31115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:27.816589 31112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:27.816422 31110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:27.817054 31102 server_base.cc:1034] running on GCE node
I20260808 15:56:27.817572 31102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:27.818539 31102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:27.819693 31102 hybrid_clock.cc:648] HybridClock initialized: now 1786204587819669 us; error 37 us; skew 500 ppm
Aug 08 15:56:27 dist-test-slave-sb2z krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:27.822449 31102 init.cc:377] Logged in from keytab as kudu/127.30.76.126@KRBTEST.COM (short username kudu)
I20260808 15:56:27.823236 31102 webserver.cc:460] Webserver started at http://127.30.76.126:44841/ using document root <none> and password file <none>
I20260808 15:56:27.823787 31102 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:27.823851 31102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:27.824008 31102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:27.825922 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data/instance:
uuid: "5899a2aeed6c40afbeadf44c76e1027e"
format_stamp: "Formatted at 2026-08-08 15:56:27 on dist-test-slave-sb2z"
server_key: "9b7755ba8930a9ada339166f5129b288"
server_key_iv: "13835d73ba61c6b0793bf43824f47bb4"
server_key_version: "encryptionkey@0"
I20260808 15:56:27.826422 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal/instance:
uuid: "5899a2aeed6c40afbeadf44c76e1027e"
format_stamp: "Formatted at 2026-08-08 15:56:27 on dist-test-slave-sb2z"
server_key: "9b7755ba8930a9ada339166f5129b288"
server_key_iv: "13835d73ba61c6b0793bf43824f47bb4"
server_key_version: "encryptionkey@0"
I20260808 15:56:27.829931 31102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 15:56:27.832155 31125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:27.833343 31102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:56:27.833478 31102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal
uuid: "5899a2aeed6c40afbeadf44c76e1027e"
format_stamp: "Formatted at 2026-08-08 15:56:27 on dist-test-slave-sb2z"
server_key: "9b7755ba8930a9ada339166f5129b288"
server_key_iv: "13835d73ba61c6b0793bf43824f47bb4"
server_key_version: "encryptionkey@0"
I20260808 15:56:27.833578 31102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:27.853762 31102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:27.856557 31102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:27.856772 31102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:27.863818 31102 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.126:39573
I20260808 15:56:27.864133 31205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.126:39573 every 8 connection(s)
I20260808 15:56:27.864774 31102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/data/info.pb
I20260808 15:56:27.867358 31206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:27.870903 31206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:27.872360 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31102
I20260808 15:56:27.872512 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-0/wal/instance
I20260808 15:56:27.872833 31025 external_mini_cluster.cc:1428] Setting key b15d7f90a31a838789133c457b0398a2
Aug 08 15:56:27 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.76.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.125@KRBTEST.COM" created.
I20260808 15:56:27.885887 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:27.873141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39573 (local address 127.0.0.1:33930)
0808 15:56:27.873696 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:27.873731 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:27.874514 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:27.875180 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:27.875184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:27.875654 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:27.875909 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:27.875921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:27.877032 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:27.877036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:27.877820 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:27.877830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:27.878434 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:27.878438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:27.878637 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:27.879186 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:27.879202 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:27.881014 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:27.882887 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:27.882897 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:27.882906 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:27.883124 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:27.883449 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:27.883454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:27.883456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:27.883620 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:27.884285 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:27.884292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:27.884434 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 15:56:27.884623 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 15:56:27.884971 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":151,"thread_start_us":153,"threads_started":1}
I20260808 15:56:27.885898 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:27.873140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.30.76.126:39573)
0808 15:56:27.873553 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:27.873572 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:27.873833 (+ 261us) server_negotiation.cc:407] Connection header received
0808 15:56:27.874800 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:27.874805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:27.874991 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:27.875178 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:27.876054 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:27.876908 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:27.877934 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:27.878288 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:27.878545 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:27.881168 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:27.881173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:27.881183 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:27.881204 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:27.882781 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:27.883237 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:27.883247 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:27.883249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:27.883353 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:27.883715 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:27.883718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:27.883720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:27.884113 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:27.884322 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:27.884709 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 15:56:27.884920 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":266,"thread_start_us":130,"threads_started":1}
I20260808 15:56:27.888396 31206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:27.889060 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:27.888826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44498)
0808 15:56:27.888934 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:27.889005 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:56:27.889310 31130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
I20260808 15:56:27.890205 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:27.890020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44510)
0808 15:56:27.890118 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:27.890160 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:56:27.890331 31206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:40409: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab.
Entry for principal kudu/127.30.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab.
I20260808 15:56:27.900635 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:40409
--webserver_interface=127.30.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab
--principal=kudu/127.30.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39573,127.30.76.125:40409,127.30.76.124:44999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:56:27.939563 31206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } attempt: 1
I20260808 15:56:27.940487 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:27.940119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44512)
0808 15:56:27.940300 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:27.940385 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 15:56:27.941298 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:27.941035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44516)
0808 15:56:27.941156 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:27.941213 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:56:27.941442 31206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:40409: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
I20260808 15:56:28.009635 31206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } attempt: 2
I20260808 15:56:28.010594 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.010278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44526)
0808 15:56:28.010445 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.010512 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 15:56:28.011369 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.011143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44532)
0808 15:56:28.011260 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.011312 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:56:28.011502 31206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.125:40409: Network error: Client connection negotiation failed: client connection to 127.30.76.125:40409: connect: Connection refused (error 111)
W20260808 15:56:28.014995 31214 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:56:28.015272 31214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:28.015322 31214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:28.015420 31214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:28.019483 31214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:28.019593 31214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:28.019631 31214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:28.019666 31214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:28.019699 31214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:28.024891 31214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39573,127.30.76.125:40409,127.30.76.124:44999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.125.keytab
--principal=kudu/127.30.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.125:40409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:28.025900 31214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:28.026922 31214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:28.035378 31224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:28.035461 31222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:28.035391 31221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:28.035804 31214 server_base.cc:1034] running on GCE node
I20260808 15:56:28.036337 31214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:28.037436 31214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:28.038686 31214 hybrid_clock.cc:648] HybridClock initialized: now 1786204588038654 us; error 36 us; skew 500 ppm
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:28.041548 31214 init.cc:377] Logged in from keytab as kudu/127.30.76.125@KRBTEST.COM (short username kudu)
I20260808 15:56:28.042357 31214 webserver.cc:460] Webserver started at http://127.30.76.125:42001/ using document root <none> and password file <none>
I20260808 15:56:28.042900 31214 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:28.042989 31214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:28.043210 31214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:28.045209 31214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data/instance:
uuid: "17a0dce8785d439381ce9fb44eac7a02"
format_stamp: "Formatted at 2026-08-08 15:56:28 on dist-test-slave-sb2z"
server_key: "360a6139d58a92f9e64781cafb787635"
server_key_iv: "4c6f215cbfbab71af4b9db265fd08eb4"
server_key_version: "encryptionkey@0"
I20260808 15:56:28.045763 31214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal/instance:
uuid: "17a0dce8785d439381ce9fb44eac7a02"
format_stamp: "Formatted at 2026-08-08 15:56:28 on dist-test-slave-sb2z"
server_key: "360a6139d58a92f9e64781cafb787635"
server_key_iv: "4c6f215cbfbab71af4b9db265fd08eb4"
server_key_version: "encryptionkey@0"
I20260808 15:56:28.049311 31214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:56:28.051496 31234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:28.052659 31214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:56:28.052793 31214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal
uuid: "17a0dce8785d439381ce9fb44eac7a02"
format_stamp: "Formatted at 2026-08-08 15:56:28 on dist-test-slave-sb2z"
server_key: "360a6139d58a92f9e64781cafb787635"
server_key_iv: "4c6f215cbfbab71af4b9db265fd08eb4"
server_key_version: "encryptionkey@0"
I20260808 15:56:28.052896 31214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:28.071369 31214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:28.073993 31214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:28.074189 31214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:28.080983 31214 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.125:40409
I20260808 15:56:28.081255 31311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.125:40409 every 8 connection(s)
I20260808 15:56:28.081918 31214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/data/info.pb
I20260808 15:56:28.084604 31312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:28.087054 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31214
I20260808 15:56:28.087205 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-1/wal/instance
I20260808 15:56:28.087481 31025 external_mini_cluster.cc:1428] Setting key 1c204b13ffa0b8d3cc6dabe0d1525c1f
I20260808 15:56:28.088280 31312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.76.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.76.124@KRBTEST.COM" created.
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:56:28.102675 31206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } attempt: 3
I20260808 15:56:28.105252 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.090208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33944 (local address 127.30.76.126:39573)
0808 15:56:28.090364 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.090369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.091115 (+ 746us) server_negotiation.cc:407] Connection header received
0808 15:56:28.092025 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.092031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.092100 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.092198 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.093373 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.094274 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.095417 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.095848 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.095970 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.100588 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.100595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.100601 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.100636 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.102742 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.103414 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.103421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.103425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.103553 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.104028 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.104034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.104039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.104359 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.104451 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.104956 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.105076 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":56}
I20260808 15:56:28.106156 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.090563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39573 (local address 127.0.0.1:33944)
0808 15:56:28.090974 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.091013 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.091828 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.092340 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.092347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.092678 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.093204 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.093225 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.094427 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.094430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.095293 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.095303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.095989 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.095992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.096125 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.096750 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.096765 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.100423 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.102861 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.102870 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.102886 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.103270 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.103697 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.103703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.103708 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.103891 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.104464 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.104474 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.104711 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.104893 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.105204 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":104,"threads_started":1}
I20260808 15:56:28.107445 31312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
I20260808 15:56:28.113276 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.103098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44546)
0808 15:56:28.103302 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.103317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.103392 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.103937 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.103940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.104119 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.104272 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.104280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.105923 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.105927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.106693 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.106707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.107581 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.107585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.107683 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.108122 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.108146 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.109624 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.111560 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.111564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.111566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.111767 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.112096 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.112098 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.112099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.112188 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.112708 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.112714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.112820 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.112939 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.113076 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":2}
I20260808 15:56:28.113368 31321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.103185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.30.76.125:40409)
0808 15:56:28.103601 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.103611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.103632 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:56:28.103670 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.103674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.103815 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.103949 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.104490 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.105781 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.106818 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.107381 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.107566 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.109745 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.109753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.109780 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.109808 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.111399 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.111884 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.111889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.111890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.111993 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.112280 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.112284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.112286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.112594 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.112744 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.113018 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.113138 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":81,"threads_started":1}
I20260808 15:56:28.115442 31206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:28.116195 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.115914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50092)
0808 15:56:28.116038 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.116101 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 15:56:28.116943 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.116699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50100)
0808 15:56:28.116830 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.116875 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 15:56:28.117081 31206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:44999: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab.
Entry for principal kudu/127.30.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab.
I20260808 15:56:28.117859 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.107981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44558)
0808 15:56:28.108729 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.108747 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.108836 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.109103 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.109106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.109278 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.109394 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.109400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.110255 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.110258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.110882 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.110896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.111506 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.111509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.111620 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.112039 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.112055 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.113596 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.116134 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.116138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.116139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.116388 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.116713 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.116715 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.116717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.116793 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.117212 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.117216 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.117316 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.117434 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.117577 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 15:56:28.117862 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.108153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.30.76.125:40409)
0808 15:56:28.108566 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.108580 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.108817 (+ 237us) server_negotiation.cc:407] Connection header received
0808 15:56:28.108911 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.108915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.108972 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.109072 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.109502 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.110140 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.110989 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.111392 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.111506 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.114386 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.114394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.114398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.114426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.116018 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.116512 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.116516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.116518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.116613 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.116895 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.116902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.116906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.117109 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.117217 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.117487 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.117691 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":325,"thread_start_us":180,"threads_started":1}
I20260808 15:56:28.118551 31312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:28.118705 31025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2MyjDc/build/debug/bin/kudu
/tmp/dist-test-task2MyjDc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:44999
--webserver_interface=127.30.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab
--principal=kudu/127.30.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.76.84:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.76.126:39573,127.30.76.125:40409,127.30.76.124:44999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:56:28.119206 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.118981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50108)
0808 15:56:28.119092 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.119157 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:56:28.119426 31240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
I20260808 15:56:28.120311 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.120071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50124)
0808 15:56:28.120169 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.120237 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 15:56:28.120529 31312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:44999: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
I20260808 15:56:28.148238 31206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } attempt: 1
I20260808 15:56:28.148993 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.148725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50132)
0808 15:56:28.148867 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.148922 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 15:56:28.149683 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.149470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50138)
0808 15:56:28.149576 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.149627 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 15:56:28.149843 31206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:44999: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
I20260808 15:56:28.169893 31312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } attempt: 1
I20260808 15:56:28.170799 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.170493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50140)
0808 15:56:28.170655 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.170720 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 15:56:28.171586 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.171333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50152)
0808 15:56:28.171441 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.171490 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:56:28.171725 31312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:44999: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
I20260808 15:56:28.224980 31206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } attempt: 2
I20260808 15:56:28.225800 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.225507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50164)
0808 15:56:28.225657 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.225723 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 15:56:28.226531 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.226325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50174)
0808 15:56:28.226430 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.226481 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 15:56:28.226711 31206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:44999: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
W20260808 15:56:28.233853 31326 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:56:28.234133 31326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:56:28.234185 31326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:56:28.234261 31326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:56:28.238395 31326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:56:28.238448 31326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:56:28.238487 31326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:56:28.238519 31326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:56:28.238569 31326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:56:28.239867 31312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } attempt: 2
I20260808 15:56:28.240543 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.240285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50180)
0808 15:56:28.240416 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.240476 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 15:56:28.241274 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.241068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50186)
0808 15:56:28.241170 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.241218 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:56:28.241509 31312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.76.124:44999: Network error: Client connection negotiation failed: client connection to 127.30.76.124:44999: connect: Connection refused (error 111)
I20260808 15:56:28.244460 31326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.76.84:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.76.126:39573,127.30.76.125:40409,127.30.76.124:44999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/krb5kdc/kudu_127.30.76.124.keytab
--principal=kudu/127.30.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.76.124:44999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 15:01:44 UTC on ccycloud.quasar-hllaxn.root.comops.site
build id 4574922
I20260808 15:56:28.245394 31326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:56:28.246325 31326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:56:28.254406 31336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:28.254397 31334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:56:28.254396 31339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:56:28.254827 31326 server_base.cc:1034] running on GCE node
I20260808 15:56:28.255401 31326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:56:28.256517 31326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:56:28.257696 31326 hybrid_clock.cc:648] HybridClock initialized: now 1786204588257687 us; error 29 us; skew 500 ppm
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:56:28.260488 31326 init.cc:377] Logged in from keytab as kudu/127.30.76.124@KRBTEST.COM (short username kudu)
I20260808 15:56:28.261312 31326 webserver.cc:460] Webserver started at http://127.30.76.124:42131/ using document root <none> and password file <none>
I20260808 15:56:28.261833 31326 fs_manager.cc:359] Metadata directory not provided
I20260808 15:56:28.261896 31326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:56:28.262058 31326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:56:28.263994 31326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data/instance:
uuid: "6b2dfe6583d54d778decc628bc41c17a"
format_stamp: "Formatted at 2026-08-08 15:56:28 on dist-test-slave-sb2z"
server_key: "5e5e4f93ab4652c0e7622e960822c3c8"
server_key_iv: "4a91d03029cb370c2332a6426d9129f3"
server_key_version: "encryptionkey@0"
I20260808 15:56:28.264495 31326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal/instance:
uuid: "6b2dfe6583d54d778decc628bc41c17a"
format_stamp: "Formatted at 2026-08-08 15:56:28 on dist-test-slave-sb2z"
server_key: "5e5e4f93ab4652c0e7622e960822c3c8"
server_key_iv: "4a91d03029cb370c2332a6426d9129f3"
server_key_version: "encryptionkey@0"
I20260808 15:56:28.267931 31326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:56:28.270074 31348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:56:28.271167 31326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:56:28.271262 31326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data,/tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal
uuid: "6b2dfe6583d54d778decc628bc41c17a"
format_stamp: "Formatted at 2026-08-08 15:56:28 on dist-test-slave-sb2z"
server_key: "5e5e4f93ab4652c0e7622e960822c3c8"
server_key_iv: "4a91d03029cb370c2332a6426d9129f3"
server_key_version: "encryptionkey@0"
I20260808 15:56:28.271340 31326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:56:28.286295 31326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:56:28.289412 31326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:56:28.289562 31326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:56:28.296057 31326 rpc_server.cc:307] RPC server started. Bound to: 127.30.76.124:44999
I20260808 15:56:28.296362 31419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.76.124:44999 every 8 connection(s)
I20260808 15:56:28.296986 31326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/data/info.pb
I20260808 15:56:28.299652 31420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:56:28.303577 31420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:28.305644 31025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2MyjDc/build/debug/bin/kudu as pid 31326
I20260808 15:56:28.305799 31025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2MyjDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786204294980517-31025-0/minicluster-data/master-2/wal/instance
I20260808 15:56:28.306140 31025 external_mini_cluster.cc:1428] Setting key 747465b9816c78eacd4804bc2208e9e2
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:56:28.317662 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.305425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.30.76.126:39573)
0808 15:56:28.305559 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.305562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.306616 (+ 1054us) server_negotiation.cc:407] Connection header received
0808 15:56:28.307883 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.307886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.307932 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.308012 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.308968 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.309474 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.310559 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.310900 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.310977 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.313662 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.313668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.313674 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.313703 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.315312 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.316001 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.316003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.316004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.316078 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.316404 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.316407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.316409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.316675 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.316788 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.317349 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.317475 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:56:28.318506 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.306847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.30.76.124:44999)
0808 15:56:28.307249 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.307263 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.307334 (+ 71us) server_negotiation.cc:407] Connection header received
0808 15:56:28.307657 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.307680 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.307936 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.308165 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.308862 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.309910 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.310827 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.311325 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.311440 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.314137 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.314143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.314152 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.314172 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.315915 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.316549 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.316557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.316559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.316669 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.316949 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.316952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.316954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.317197 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.317381 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.317596 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.317727 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":294,"thread_start_us":117,"threads_started":1}
I20260808 15:56:28.318506 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.305885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.126:39573 (local address 127.0.0.1:33958)
0808 15:56:28.306427 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.306470 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.307657 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.308083 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.308093 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.308548 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.308811 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.308831 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.309578 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.309582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.310443 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.310452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.311028 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.311032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.311208 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.311928 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.311942 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.313518 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.315419 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.315427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.315435 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.315908 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.316161 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.316164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.316165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.316287 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.316784 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.316790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.317057 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.317263 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.317574 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":36,"thread_start_us":171,"threads_started":1}
I20260808 15:56:28.319757 31420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.126@KRBTEST.COM
I20260808 15:56:28.325861 31206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } attempt: 3
I20260808 15:56:28.328959 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.320220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.125:40409 (local address 127.0.0.1:44568)
0808 15:56:28.320444 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.320458 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.320539 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.321012 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.321015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.321212 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.321342 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.321350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.322301 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.322303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.323059 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.323069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.323610 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.323617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.323694 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.324107 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.324124 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.325426 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.327401 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.327404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.327406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.327664 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.327940 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.327944 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.327946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.328036 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.328503 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.328505 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.328587 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.328679 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.328792 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 15:56:28.329056 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.320330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.30.76.125:40409)
0808 15:56:28.320443 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.320449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.320502 (+ 53us) server_negotiation.cc:407] Connection header received
0808 15:56:28.320617 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.320620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.320680 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.320780 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.321476 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.322192 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.323173 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.323456 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.323612 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.325533 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.325537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.325540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.325567 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.327290 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.327761 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.327764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.327766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.327850 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.328157 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.328160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.328162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.328361 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.328431 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.328750 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.328875 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 15:56:28.330192 31420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:56:28.330803 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.320780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.30.76.126:39573)
0808 15:56:28.320890 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.320894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.320906 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:56:28.320972 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.320973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.321003 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.321062 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.321737 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.322406 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.323380 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.323710 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.323799 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.326209 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.326215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.326221 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.326248 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.328514 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.328982 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.328986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.328988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.329071 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.329380 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.329384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.329385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.329607 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.329830 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.330531 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.330659 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.126@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:56:28.332672 31312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } attempt: 3
I20260808 15:56:28.334878 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.326215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50206)
0808 15:56:28.326329 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.326347 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.326421 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.326866 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.326869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.327066 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.327184 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.327191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.328001 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.328004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.328767 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.328779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.329377 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.329380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.329460 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.329880 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.329894 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.331302 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.333375 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.333378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.333380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.333609 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.333912 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.333915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.333917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.334001 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.334424 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.334428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.334509 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.334597 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.334731 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 15:56:28.335428 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.326465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.30.76.124:44999)
0808 15:56:28.326630 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.326635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.326649 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:56:28.326688 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.326693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.326745 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.326828 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.327295 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.327914 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.328892 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.329254 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.329678 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.331441 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.331444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.331447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.331472 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.333255 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.333725 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.333728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.333730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.333813 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.334102 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.334106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.334107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.334287 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.334380 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.335183 (+ 803us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.335292 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.124@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.76.125@KRBTEST.COM for kudu/127.30.76.124@KRBTEST.COM
I20260808 15:56:28.339470 31206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: Bootstrap starting.
Aug 08 15:56:28 dist-test-slave-sb2z krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786204587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.76.125@KRBTEST.COM
I20260808 15:56:28.342259 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.330654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50210)
0808 15:56:28.331268 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.331281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.331368 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.331617 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.331623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.332062 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.332171 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.332180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.333892 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.333896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.334641 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.334650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.335331 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.335335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.335406 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.335988 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.335999 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.337291 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.340237 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.340239 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.340241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.340484 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.340752 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.340755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.340757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.340840 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.341390 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.341392 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.341462 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.341550 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.342083 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554}
I20260808 15:56:28.342504 31206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:28.342815 31429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.330709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50210 (local address 127.30.76.124:44999)
0808 15:56:28.331120 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.331128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.331350 (+ 222us) server_negotiation.cc:407] Connection header received
0808 15:56:28.331442 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.331445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.331484 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.331603 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.332525 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.333783 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.334751 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.335146 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.335237 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.337427 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.337430 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.337433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.337454 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.340115 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.340579 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.340582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.340583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.340666 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.340944 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.340947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.340949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.341270 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.341359 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.341664 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.342633 (+ 969us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":855,"server-negotiator.queue_time_us":327,"thread_start_us":112,"threads_started":1}
I20260808 15:56:28.343855 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.333009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.76.124:44999 (local address 127.0.0.1:50218)
0808 15:56:28.333212 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 15:56:28.333226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:56:28.333302 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:56:28.334028 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:56:28.334031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:56:28.334205 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:56:28.334284 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.334293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.335403 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.335406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.336108 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:56:28.336118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.337279 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.337282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:56:28.337386 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.337880 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:56:28.337899 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:56:28.339339 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:56:28.342029 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.342032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.342034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.342344 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.342657 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:56:28.342662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:56:28.342663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:56:28.342744 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:56:28.343195 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:56:28.343199 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:56:28.343326 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 15:56:28.343428 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 15:56:28.343696 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":3}
I20260808 15:56:28.344033 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.332379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.30.76.125:40409)
0808 15:56:28.332511 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.332515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.332527 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:56:28.332612 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.332616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.332662 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.332731 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.333560 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.334260 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.335358 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.336492 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.336592 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.340356 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.340360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.340363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.340393 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.342107 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.342603 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.342609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.342610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.342697 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.343020 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.343024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.343025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.343240 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.343366 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.343759 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.343871 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":667,"server-negotiator.queue_time_us":61}
I20260808 15:56:28.344064 31206 log.cc:824] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:28.345566 31420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: Bootstrap starting.
I20260808 15:56:28.345870 31206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: No bootstrap required, opened a new log
I20260808 15:56:28.347404 31420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:28.348124 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.333546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.30.76.124:44999)
0808 15:56:28.333813 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.333820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.333832 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:56:28.333863 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.333865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.333902 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.334391 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.334514 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.335290 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.336732 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.337151 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.337641 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.339491 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.339496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.339501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.339575 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.341914 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.342449 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.342453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.342454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.342540 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.342857 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.342864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.342867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.343075 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.343166 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.343506 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.347959 (+ 4453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":116,"threads_started":1}
I20260808 15:56:28.348235 31420 log.cc:824] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:56:28.349440 31206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.349763 31206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5899a2aeed6c40afbeadf44c76e1027e, State: Initialized, Role: FOLLOWER
I20260808 15:56:28.350111 31206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.350551 31433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.350737 31433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:28.350901 31025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:56:28.351148 31312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: Bootstrap starting.
I20260808 15:56:28.351413 31206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:28.352131 31166 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5899a2aeed6c40afbeadf44c76e1027e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974
I20260808 15:56:28.352358 31166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 15:56:28.352463 31166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:28.353624 31312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: Neither blocks nor log segments found. Creating new log.
I20260808 15:56:28.353668 31166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.354581 31166 leader_election.cc:288] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [CANDIDATE]: Term 1 election: Requested vote from peers 17a0dce8785d439381ce9fb44eac7a02 (127.30.76.125:40409), 6b2dfe6583d54d778decc628bc41c17a (127.30.76.124:44999)
I20260808 15:56:28.355032 31420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: No bootstrap required, opened a new log
W20260808 15:56:28.357609 31454 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 15:56:28.357638 31312 log.cc:824] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: Log is configured to *not* fsync() on all Append() calls
W20260808 15:56:28.357754 31454 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 15:56:28.358384 31132 tablet.cc:2369] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:56:28.359880 31420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.360114 31420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2dfe6583d54d778decc628bc41c17a, State: Initialized, Role: FOLLOWER
I20260808 15:56:28.360441 31420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.360702 31312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: No bootstrap required, opened a new log
I20260808 15:56:28.357462 31102 master_runner.cc:186] Error getting master registration for 127.30.76.125:40409: OK, instance_id { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" instance_seqno: 1786204588079157 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:56:28.361028 31102 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:56:28.361707 31420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:28.361966 31385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5899a2aeed6c40afbeadf44c76e1027e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b2dfe6583d54d778decc628bc41c17a"
I20260808 15:56:28.362203 31385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:28.363564 31385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5899a2aeed6c40afbeadf44c76e1027e in term 1.
I20260808 15:56:28.364004 31129 leader_election.cc:302] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5899a2aeed6c40afbeadf44c76e1027e, 6b2dfe6583d54d778decc628bc41c17a; no voters:
I20260808 15:56:28.364300 31433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:56:28.364519 31433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [term 1 LEADER]: Becoming Leader. State: Replica: 5899a2aeed6c40afbeadf44c76e1027e, State: Running, Role: LEADER
I20260808 15:56:28.364846 31433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.364933 31312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.365144 31312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a0dce8785d439381ce9fb44eac7a02, State: Initialized, Role: FOLLOWER
I20260808 15:56:28.365446 31312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } }
I20260808 15:56:28.366504 31433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5899a2aeed6c40afbeadf44c76e1027e. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.366653 31433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:28.366729 31312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:56:28.366767 31276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5899a2aeed6c40afbeadf44c76e1027e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "17a0dce8785d439381ce9fb44eac7a02"
I20260808 15:56:28.366969 31276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:56:28.368464 31276 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5899a2aeed6c40afbeadf44c76e1027e in term 1.
I20260808 15:56:28.371803 31489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.371937 31489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 15:56:28.372961 31509 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:28.373039 31509 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:28.375694 31465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.375825 31465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: This master's current role is: FOLLOWER
W20260808 15:56:28.377090 31487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:56:28.377182 31487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:56:28.377480 31493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:56:28.378834 31493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:56:28.382786 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.358946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.30.76.124:44999)
0808 15:56:28.359045 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.359048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.361471 (+ 2423us) server_negotiation.cc:407] Connection header received
0808 15:56:28.361744 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.361747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.361796 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.361850 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.368290 (+ 6440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.369035 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.370166 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.370488 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.370564 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.377064 (+ 6500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.377068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.377071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.377096 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.378882 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.379950 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.379954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.379956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.380046 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.380373 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.380376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.380378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.380909 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.381086 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.382484 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.382614 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 15:56:28.389658 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.358476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.30.76.126:39573)
0808 15:56:28.358629 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.358634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.371449 (+ 12815us) server_negotiation.cc:407] Connection header received
0808 15:56:28.371657 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.371660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.371706 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.371756 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.375142 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.375871 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.381419 (+ 5548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.381762 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.381839 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.385411 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.385415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.385417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.385443 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.387739 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.388248 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.388254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.388258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.388338 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.388667 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.388670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.388671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.388887 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.388999 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.389359 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.389470 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 15:56:28.389897 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:28.359322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.30.76.125:40409)
0808 15:56:28.359420 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.359424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.368129 (+ 8705us) server_negotiation.cc:407] Connection header received
0808 15:56:28.368264 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.368270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.368318 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.368389 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.373197 (+ 4808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.373964 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.379734 (+ 5770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:28.380075 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:28.380161 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:28.386340 (+ 6179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:56:28.386347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:56:28.386354 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:56:28.386389 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:56:28.388065 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.388562 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.388569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.388573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.388662 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:56:28.388969 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:56:28.388973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:56:28.388974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:56:28.389198 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:56:28.389307 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:28.389630 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 15:56:28.389752 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 15:56:28.400738 31276 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [term 1 FOLLOWER]: Refusing update from remote peer 5899a2aeed6c40afbeadf44c76e1027e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:56:28.402236 31385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [term 1 FOLLOWER]: Refusing update from remote peer 5899a2aeed6c40afbeadf44c76e1027e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:56:28.402994 31433 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:56:28.403692 31488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:56:28.407588 31489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5899a2aeed6c40afbeadf44c76e1027e. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.407694 31489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:28.410665 31465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5899a2aeed6c40afbeadf44c76e1027e. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.410768 31465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:28.419234 31488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.419358 31488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:28.419898 31511 mvcc.cc:205] Tried to move back new op lower bound from 7316293994082340864 to 7316293993945665536. Current Snapshot: MvccSnapshot[applied={T|T < 7316293994082340864}]
I20260808 15:56:28.421584 31465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.421686 31465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:28.423075 31433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.423195 31433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e [sys.catalog]: This master's current role is: LEADER
I20260808 15:56:28.424111 31489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5899a2aeed6c40afbeadf44c76e1027e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899a2aeed6c40afbeadf44c76e1027e" member_type: VOTER last_known_addr { host: "127.30.76.126" port: 39573 } } peers { permanent_uuid: "17a0dce8785d439381ce9fb44eac7a02" member_type: VOTER last_known_addr { host: "127.30.76.125" port: 40409 } } peers { permanent_uuid: "6b2dfe6583d54d778decc628bc41c17a" member_type: VOTER last_known_addr { host: "127.30.76.124" port: 44999 } } }
I20260808 15:56:28.424227 31489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:56:28.426604 31514 mvcc.cc:205] Tried to move back new op lower bound from 7316293994082340864 to 7316293993945665536. Current Snapshot: MvccSnapshot[applied={T|T < 7316293994082340864}]
I20260808 15:56:28.426775 31517 mvcc.cc:205] Tried to move back new op lower bound from 7316293994082340864 to 7316293993945665536. Current Snapshot: MvccSnapshot[applied={T|T < 7316293994082340864}]
I20260808 15:56:28.430159 31493 catalog_manager.cc:1347] Generated new cluster ID: 0dc91d332b144b70be18e5a19ccabf54
I20260808 15:56:28.430222 31493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:56:28.468432 31493 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:56:28.471887 31493 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:56:28.500375 31493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5899a2aeed6c40afbeadf44c76e1027e: Generated new TSK 0
I20260808 15:56:28.501367 31493 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:56:28.543025 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.539643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.30.76.125:40409)
0808 15:56:28.539789 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.539794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.541829 (+ 2035us) server_negotiation.cc:407] Connection header received
0808 15:56:28.542058 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.542062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.542117 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.542190 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.542881 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44582: BlockingRecv error: recv got EOF from 127.0.0.1:44582 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 15:56:28.544632 31430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.540544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.30.76.124:44999)
0808 15:56:28.540675 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.540680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.543623 (+ 2943us) server_negotiation.cc:407] Connection header received
0808 15:56:28.543812 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.543815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.543866 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.543922 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.544513 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50238: BlockingRecv error: recv got EOF from 127.0.0.1:50238 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 15:56:28.546446 31209 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.541276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.30.76.126:39573)
0808 15:56:28.541393 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.541398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.545273 (+ 3875us) server_negotiation.cc:407] Connection header received
0808 15:56:28.545617 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.545620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.545679 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.545737 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.546322 (+ 585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33988: BlockingRecv error: recv got EOF from 127.0.0.1:33988 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 15:56:28.551172 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.549627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44596 (local address 127.30.76.125:40409)
0808 15:56:28.549743 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.549747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.550211 (+ 464us) server_negotiation.cc:407] Connection header received
0808 15:56:28.550425 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.550429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.550506 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.550560 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.551052 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44596: BlockingRecv error: recv got EOF from 127.0.0.1:44596 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 15:56:28.556303 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.554867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.30.76.125:40409)
0808 15:56:28.554957 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.554961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.555400 (+ 439us) server_negotiation.cc:407] Connection header received
0808 15:56:28.555601 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.555605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.555653 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.555702 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.556214 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44608: BlockingRecv error: recv got EOF from 127.0.0.1:44608 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 15:56:28.561929 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.560534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.30.76.125:40409)
0808 15:56:28.560624 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.560628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.561123 (+ 495us) server_negotiation.cc:407] Connection header received
0808 15:56:28.561304 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.561307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.561350 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.561397 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.561838 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44624: BlockingRecv error: recv got EOF from 127.0.0.1:44624 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 15:56:28.569588 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.567990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.30.76.125:40409)
0808 15:56:28.568114 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.568118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.568618 (+ 500us) server_negotiation.cc:407] Connection header received
0808 15:56:28.568799 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.568803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.568851 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.568915 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.569489 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44630: BlockingRecv error: recv got EOF from 127.0.0.1:44630 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 15:56:28.581252 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.579698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.30.76.125:40409)
0808 15:56:28.579806 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.579811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.580389 (+ 578us) server_negotiation.cc:407] Connection header received
0808 15:56:28.580587 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.580590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.580654 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.580743 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.581150 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44642: BlockingRecv error: recv got EOF from 127.0.0.1:44642 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 15:56:28.599857 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.598833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44644 (local address 127.30.76.125:40409)
0808 15:56:28.598959 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.598962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.599103 (+ 141us) server_negotiation.cc:407] Connection header received
0808 15:56:28.599220 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.599222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.599267 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.599348 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.599770 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44644: BlockingRecv error: recv got EOF from 127.0.0.1:44644 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 15:56:28.635149 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.633979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.30.76.125:40409)
0808 15:56:28.634130 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.634134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.634280 (+ 146us) server_negotiation.cc:407] Connection header received
0808 15:56:28.634445 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.634449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.634502 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.634588 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.635028 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44652: BlockingRecv error: recv got EOF from 127.0.0.1:44652 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 15:56:28.702663 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.701400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.30.76.125:40409)
0808 15:56:28.701548 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.701553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.701741 (+ 188us) server_negotiation.cc:407] Connection header received
0808 15:56:28.701954 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.701957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.702014 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.702094 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.702516 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44666: BlockingRecv error: recv got EOF from 127.0.0.1:44666 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 15:56:28.834219 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:28.832944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44674 (local address 127.30.76.125:40409)
0808 15:56:28.833122 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:28.833126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:28.833296 (+ 170us) server_negotiation.cc:407] Connection header received
0808 15:56:28.833470 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:28.833473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:28.833528 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:28.833619 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:28.834086 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44674: BlockingRecv error: recv got EOF from 127.0.0.1:44674 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 15:56:29.094059 31322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:56:29.092494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.30.76.125:40409)
0808 15:56:29.092660 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:29.092664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:29.092826 (+ 162us) server_negotiation.cc:407] Connection header received
0808 15:56:29.093006 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:29.093011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:29.093138 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:29.093242 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:56:29.093932 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44678: BlockingRecv error: recv got EOF from 127.0.0.1:44678 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:56:29.374249 31509 catalog_manager.cc:1259] Loaded cluster ID: 0dc91d332b144b70be18e5a19ccabf54
I20260808 15:56:29.374351 31509 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: loading cluster ID for follower catalog manager: success
I20260808 15:56:29.376202 31509 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: acquiring CA information for follower catalog manager: success
I20260808 15:56:29.377388 31509 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17a0dce8785d439381ce9fb44eac7a02: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:56:29.378177 31487 catalog_manager.cc:1259] Loaded cluster ID: 0dc91d332b144b70be18e5a19ccabf54
I20260808 15:56:29.378232 31487 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: loading cluster ID for follower catalog manager: success
I20260808 15:56:29.379663 31487 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: acquiring CA information for follower catalog manager: success
I20260808 15:56:29.380602 31487 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b2dfe6583d54d778decc628bc41c17a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:56:29.612005 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:29.608182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.30.76.125:40409)
0808 15:56:29.608431 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:29.608435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:29.608452 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:56:29.608585 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:29.608588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:29.608716 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:29.608817 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:29.609047 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.609561 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.610370 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.610597 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.610676 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:29.611066 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:29.611574 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:29.611695 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:29.611741 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 15:56:29.611791 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
I20260808 15:56:29.618273 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:29.614889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.30.76.124:44999)
0808 15:56:29.615107 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:29.615111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:29.615140 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:56:29.615335 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:29.615339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:29.615484 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:29.615632 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:29.615809 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.616300 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.617101 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.617306 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.617377 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:29.617619 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:29.617976 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:29.618058 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:29.618115 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 15:56:29.618164 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260808 15:56:29.625330 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:29.621466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.30.76.125:40409)
0808 15:56:29.621593 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:29.621597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:29.621799 (+ 202us) server_negotiation.cc:407] Connection header received
0808 15:56:29.621988 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:29.621991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:29.622034 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:29.622113 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:29.622355 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.623054 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.624090 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.624361 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.624451 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:29.624785 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:29.624865 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:29.624903 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:29.625118 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 15:56:29.625181 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:56:29.625547 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:29.621541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.30.76.124:44999)
0808 15:56:29.621650 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:29.621655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:29.621693 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:56:29.621876 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:29.621880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:29.621930 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:29.622000 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:29.622632 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.623343 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.624443 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.624652 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.624759 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:29.625091 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:29.625163 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:29.625229 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:29.625383 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 15:56:29.625434 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:56:29.626179 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:56:29.621257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.30.76.126:39573)
0808 15:56:29.621881 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 15:56:29.621886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:56:29.621898 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:56:29.621982 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:56:29.621988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:56:29.622135 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:56:29.622253 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:56:29.622462 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.623056 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.624259 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:56:29.624622 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:56:29.624747 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:56:29.625066 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:56:29.625554 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:56:29.625643 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:56:29.625979 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 15:56:29.626037 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":70,"threads_started":1}
I20260808 15:56:29.629106 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31102
I20260808 15:56:29.641566 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31214
I20260808 15:56:29.650898 31025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2MyjDc/build/debug/bin/kudu with pid 31326
2026-08-08T15:56:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3315 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6618 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (294666 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST